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Volume I: Assessment Report 

1.0 Authorization and Notification 

The assessment was requested and approved out-of-board on March 5, 2007 by authority of the 
Director, NASA Engineering and Safety Center (NESC). The NESC team was chartered by the 
NASA Office of Chief Engineer (OCE), and the Associate Administrator (AA) Space Operations 
Mission Directorate (SOMD) to assess hail damage repair efforts on the Space Transportation 
System (STS)-l 17 External Tank (ET)-124. Mr. Tim Wilson, Deputy Director NESC, was the 
responsible lead for this assessment. The four key stakeholders for this investigation are Mr. 
Wayne Hale, Space Shuttle Program (SSP) Manager at the Johnson Space Center in Houston, 
Texas; Mr. Chris Scolese, NASA Chief Engineer at NASA’s Headquarters in Washington, D.C.; 
Mr. William Gerstenmaier, AA SOMD at NASA’s Headquarters in Washington, D.C.; and Mr. 
Bryan O’Connor, Chief Safety and Mission Assurance (S&MA) Officer at NASA Headquarters 
in Washington, D.C. 
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4.0 Executive Summary 

Severe thunderstorms with associated hail and high winds struck the STS-1 17 stack on February 
26, 2007. Peak winds were recorded at 62 knots with hail sizes ranging from 0.3 inch to 0.8 inch 
in diameter. As a result of the storm, the North Carolina Foam Institute (NCFI) type 24-124 
Thermal Protection System (TPS) foam on the liquid oxygen (L02) ogive acreage incurred 
significant impact damage. The NCFI on the ET intertank and the liquid hydrogen (LH2) acreage 
sustained hail damage. The Polymer Development Laboratory (PDL)-1034 foam of the L02 ice 
frost ramps (IFRs) and the Super-Lightweight Ablator (SLA) of the L02 cable tray also suffered 
minor damage. 

NESC was asked to assess the technical feasibility of repairing the ET TPS, the reasonableness 
of conducting those repairs with the vehicle in a vertical, integrated configuration at the Kennedy 
Space Center (KSC) Vehicle Assemble Building (VAB), and to address attendant human factors 
considerations including worker fatigue and the potential for error. As the assessment 
progressed, the latter question evolved to a human factors assessment of the KSC work 
environment and adequacy of the process controls applied to PDL-type standard repairs. 

ET-124 was subjected to a comprehensive visual inspection and repairs were executed to criteria 
documented in the ET Post-Build Acceptance and In-Process Re-Work Requirements Manual, 
Offsite, 80901019010 (9010). The 9010 requirements were modified in some instances to 
accommodate the high volume of repairs required for ET-124, and NESC reviewed and 
concurred with these modifications. The bulk of the damage was assessed and processed in one 
of three standard-repair categories: “use-as-is,” (minimal or no rework required), “sand-and- 
blend”or“PDL pour.” Hand-packed SLA was used to repair damage to the cable trays. Two 
areas of the L02 tank, one just aft of the composite nose cap and the other in the aft section, 
were repaired using a non-standard BX-265 manual spray over the NCFI 24-124. This non- 
standard repair technique is similar to that used where the Protuberance Air Load (PAL) ramps 
were removed from the intertank and the L02 and LH2 tanks. 

NESC participated in technical discussions surrounding the inspections, repair categorization, 
and the repair processes and reviewed supporting planning, testing, and analyses. Key questions 
concerning the debris potential posed by undetected crushed foam remaining on ET-124 and 
adequacy of the various repair techniques were addressed. Recommendations were forwarded to 
the ET Project as the work progressed and all but one of the recommendations was implemented. 
NESC concurred with flying ET-124 on the STS-1 17 mission as-repaired. 
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5.0 Assessment Plan 

This assessment was conducted to evaluate the STS-1 17 ET-124 TPS hail damage repair effort. 
The task had three primary elements: (1) Evaluate the technical feasibility of repairing the 
damaged TPS foam, (2) assess the reasonableness of conducting those repairs with the vehicle in 
a vertical, integrated configuration at the KSC VAB, and (3) address attendant human factors 
considerations including worker fatigue and the potential for error. As the assessment 
progressed, the latter question evolved to a human factors assessment of the KSC work 
environment and adequacy of the process controls applied to PDL-type standard repairs. 

Team members participated in planning and review meetings and maintained informal 
communication with ET Project management. The team participated in several Technical 
Interchange Meetings (TIM) and relevant ET Project review boards, most notably the ET Chief 
Engineer’s Review Board (CERB). Team members evaluated the effectiveness of verification 
testing and analysis performed by the ET Project. A limited amount of independent analysis was 
performed by the NESC to confirm flight rationale for non-standard repair processes and to 
evaluate flight risk associated with prelaunch icing and thermal conditions during ascent. 
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6.0 Description of the Problem and Approach 

6.1 Problem Description 

Severe thunderstorms with associated hail and high winds struck the STS-1 17 stack on February 
26, 2007 while at Launch Pad 39. Peak winds were recorded at 62 knots with hail size ranging 
from 0.3 inch to 0.8 inch in diameter as shown in Figure 6.1-1. The NCFI 24-124 foam on the 
L02 tank ogive acreage and the ET intertank incurred significant damage. NCFI on the ET 
intertank and PDL 1034 foam of the L02 IFRs suffered minor damage, as did the L02 tank 
cable tray SLA. An example of the resulting damage is shown in Figure 6.1-2. 



Figure 6.1-1. Hail from the February 26, 
2007 storm [ref. 26] 


Figure 6.1-2. Example of Hail Damage to 
ET-124 TPS [ref. 27] 
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In the initial damage assessment [ref. 3], the ET Project reported that the nose cone and 
machined surface of the L02 ogive TPS had suffered a significant number of impacts. As Figure 
6.1-3 depicts, impact damage to ET-124 was sustained around the entire circumference with the 
heaviest concentration on the +Z side with more than 1200 hits. There were numerous areas of 
damaged foam in the L02 tank acreage TPS with the heaviest concentration in the +Z and -Y 
quadrants. Minor damage was observed on the inboard side of all L02 IFRs with crushed foam 
observed on the IFRs at stations Xt-634 and Xt-676. Numerous areas of damaged NCFI were 
observed in the intertank area with the heaviest sustained in the -Z quadrant. Additional less 
severe damage was also observed in the +Z quadrant. Fewer than 12 hits were observed on the 
LH2 acreage TPS. Minor damage was observed on the -Y ET / Solid Rocket Booster (SRB) 
ramp aft fairing and cable tray and on the forward face of the LH2 IFRs. 
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Figure 6.1-3. Initial Assessment of Hail Damage to ET-124 [ref. 4] 
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6.2 Approach 

ET Project performed detailed inspections of the tank and documented the location and extent of 
the damage. Inspections addressed all TPS surfaces and exposed composite and metallic 
components. Repair plans were developed to return ET- 124 to a configuration meeting the 
requirements documented in the ET Post-Build Acceptance and In-Process Re-Work 
Requirements Manual, Offsite, 80901019010 (referred to herein as “9010”). Some variances 
from 9010 requirements were accepted by ET Project Material Review Board (MRB) action to 
reduce the number and invasiveness of repairs and minimize removal of undamaged TPS 
material. Previously validated standard repair techniques were implemented wherever possible. 

6.2.1 Damage Inspection 

The ET-124 impact damage inspection involved visual and tactile inspection of the tank surfaces 
and defects by experienced KSC United Space Alliance (USA) and Lockheed-Martin Space 
Systems Company (LMSSC) personnel [ref. 11]. A detailed TPS inspection grid was developed 
and an attributes database establishing to catalog individual impact sites and disposition of 
repairs. A summary is presented in Table 6.2-1 [ref. 25]. 

Exposed polymer matrix composite (PMC) hardware (the nose cap, intertank access door, and 
L02 feedline fairing) were subjected to thermography non-destructive evaluation (NDE) with 
the results compared to the baseline inspection data. 
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Table 6.2-1. STS-117 Hail Damage and Disposition 


PR# 

Description 

Repair Type 

Qty 

ET-124-TS-0014 

L02 Tank TPS 

BX-265 

450 



Nodule Only 

8 



PDL Repair 

940 



Sand and Blend 

888 



Use As Repaired 

208 

ET-124-TS-0015 

+Y Aft Fairing 

Use As Repaired 

19 

ET-124-TS-0016 

-Y Aft Fairing 

Use As Repaired 

6 

ET-124-TS-0017 

SRB Pal Ramps 

Use As Repaired 

1 

ET-124-TS-0018 

Ice Frost Ramps C/O 

Use As Repaired 

6 

ET-124-TS-0019 

Ice Frost Ramps TPS 

Use As Repaired 

4 

ET-124-TS-0022 

LH2 Tank TPS 

Nodule Only 

2 



PDL Repair 

19 



Use As Repaired 

1 

ET-124-TS-0023 

Intertank TPS 

PDL Repair 

76 

ET-124-TS-0024 

L02 Intertank Flange 

Use As Repaired 

105 

ET-124-TS-0025 

LH2 Tank Flange 

Use As Repaired 

5 

ET-124-TS-0026 

Vertical Strut 

Use As Repaired 

23 

ET-124-TS-0027 

Vertical Strut Cable Tray 

Use As Repaired 

23 

ET-124-TS-0028 

Umbilical Cable Tray 

Use As Repaired 

4 


TOTAL 

2788 


6.2.2 Damage Disposition 

The flowchart in Figure 6.2-1 outlines the initial plan for dispositioning TPS damage [ref. 9]. If 
the damage was superficial (cosmetic only), no repair was performed. If the damage depth was 
less than 0.3 inch, the crushed NCFI was removed and no further repair performed. PDL 1034 
repairs were performed on damage greater than 0.3 inch in depth. Surface roughness and 
waviness criteria outlined in 9010 were modified to minimize the amount of sanding required. In 
two locations where size and density of PDL repairs would have resulted in violation of 9010 or 
pertinent Interface Control Document (ICD) requirements, large-area BX-265 spray repairs were 
performed. Minor violations of 9010 were accepted by MRB disposition on a case-by-case 
basis. 

Minimum NCFI thickness criteria were revised downward slightly to minimize the number of 
PDL repairs required in the L02 tank acreage foam. Specifically, the foam thickness criteria for 
acreage areas on ET-124 in the No-Ice and Ice Limitation zones were reduced to the minimums 
required to maintain an 85 percent launch probability between May and October. The minimum 
thickness criterion was reduced to 0.8 inch in the L02 tank No-Ice zone which included a 0.1 
inch thickness buffer to protect against any undetected crushed foam. The minimum thickness 
criterion was changed to 0.6 inch in the L02 Ice-Limitation zone which also includes a 0.1 inch 
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buffer. The minimum NCFI thickness of the LH2 tank Ice Limitation zone was changed to 0.88 
inch. 
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Figure 6.2-1. Preliminary Roadmap for ET-124 Repair Evaluation Process [ref. 9] 


6.2.3 Repair Execution 
6.2.3.1 Sand and Blend 

Crushed foam was removed from damage locations less than 0.25 inch in depth and the 
remaining NCFI thickness evaluated. Sand and blend repairs were implemented at locations 
where the remaining TPS was sufficient to meet the revised minimum foam thickness criteria. 
While the 9010 document specifies a 16: 1 diameter-to-depth ratio corresponding to a wall angle 
of approximately 7.5 degrees for repairs of this type, the criteria was modified for ET-124 to 
minimize removal of adjacent NCFI. The edges of defects up to 0.1 inch deep were slightly 
rounded. Defects greater than 0. 1 inch deep were blended by sanding foam from an area 
extending no more than about 0.5 inch from the edge of the damage site. Steeper-than-normal 
wall angles were created in both instances. In another variation from the 9010 requirements, red 
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dye was not applied to defects before the crushed NCFI was removed. If the final defect depth 
did not exceed 0.3 inch and no crack-like indications were evident, red dye was not applied after 
the sand-and-blend repair was completed. The result of these variations is an increase in the 
potential for day-of-launch ice formation at sand and blend repair locations, and an increased risk 
for undetected crushed foam at some locations. 
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Figure 6.2-2. Sand and Blend Wall Angles for Defects > 0.1” Deep [ref. 24] 

6.2.3.2 PDL Repairs 

A PDL repair was performed if the acreage TPS thickness remaining under the defect was less 
than the required TPS minimum. If damage was visible, the degraded NCFI was removed with a 
grinder and the surface was sanded and cleaned. Red dye was then applied to the defect to 
highlight any remaining damage. The process was repeated until all crushed foam was removed. 
Conathane® 1 adhesive and PDL- 1034 were applied to the repair area per standard processes, 
with the exception that up to three small co-located repairs were filled using PDL injected from a 
single syringe. Mold removal resulted in minor collateral damage at some locations which was 
removed by sanding the surrounding NCFI. PDL repairs were performed to 9010 requirements 
with the exception of an initial red dye application, which was not done prior to initial damage 
removal and defect depth evaluation. While PDL repair size (diameter, depth, and area) 
requirements were met, minor violations to 9010 repair spacing criteria were accepted by ET 
Project MRB action at 148 locations, LM IRT action response IRT-25. 

6.2.3.3 BX-265 Spray Repairs 

Non-standard BX-265 sprays were required in two areas of the L02 tank where a high 
concentration of impact damage precluded PDL repair due to repair spacing and ICD 
considerations. One of these locations was adjacent to the ET nose cone and the other in the aft 


1 Is a registered trademark of Conap, Inc. Corporation, New York 
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ogive region. The forward ogive region adjacent to the nose cone was approximately 20 inches 
in length and extended 300 degrees around the tank circumference. The second, in the aft ogive 
region, was approximately 20 inches wide by 225 inches long. In both locations, approximately 
0.5 inch of NCFI was removed from the L02 tank acreage to create a foundation for manually- 
sprayed BX-265. No evidence of residual damage was detected in either location after the NCFI 
was removed and red dye applied [ref. 23]. 

6.2.3.4 Additional Repairs 

Minor repairs to the L02 tank cable tray (SLA) were performed to 9010 requirements using 
standard repair processes. 

6.2.4 Flight Rationale 

ET Project flight rationale for ET-124 is based on verification by inspection, test, analysis, or 
similarity that repairs meet the derived requirements established in the 9010 specification. Flight 
performance of repairs conducted using equivalent processes and under similar circumstances 
offers additional confidence, especially those done to repair hail damage sustained during the 
STS-96 processing flow and woodpecker damage during STS-70. Performance of a stress relief 
groove cut in the L02 tank acreage foam and flown on STS-91 offered some visibility into the 
ascent performance of NCFI with significant waviness. The debris risk change posed by the 
sheer volume of repairs was assessed by the SSP Systems Engineering and Integration (SE&I) 
organization using the Probabilistic Risk Assessment (PRA) process established after STS- 107. 

6.2.4.1 Test and Analysis 

A number of tests and process demonstrations were conducted by the ET Project to substantiate 
flight rationale for ET-124 repair procedures. The key testing focused on crushed NCFI, PDL 
repairs, and the BX-265. Process demonstrations were required to validate the BX-265 sprays 
and the one-syringe / three-pour process used for the PDL repairs. 

6.2.4.1.1 Crushed NCFI 24-124 Foam Testing 

The objectives of the crushed NCFI 24-124 testing were to demonstrate that performance of the 
repaired and un-repaired crushed foam would be acceptable in expected pre-launch and flight 
environments. The combined environments, thermal-vacuum, and hot gas tests were meant to 
demonstrate that foam insulating properties would be preserved on the pad and during ascent, 
and that significant debris liberation would not occur in flight. Testing was conducted under 
conditions intended to simulate the heat rates expected in the LOX ogive area on ascent. Icing 
tests were intended to demonstrate that no unacceptable ice/frost buildup will occur even in foam 
that is crushed to the detectability limit (i.e., barely visible damage). 
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Extensive crushed foam testing was conducted in conjunction with the STS- 1 14 In-Flight 
Anomaly (IFA) investigation and much of this data was applicable to ET-124. The results of 
tests conducted to support development of STS-96 hail damage flight rationale were also 
available and were reviewed. Additional testing of panels with simulated hail damage was 
performed at the request of NESC. NCFI 24-124 panels were sprayed to a thickness 
representative of the high density impact damage area away from the nose cone BX-265 manual 
spray repair. The panels were impacted in such a manner representative of the STS-1 17 hail 
damage. The panels were inspected and repaired using the same techniques, processes and 
criteria planned for ET-124. Hot gas testing was then conducted to assess the potential for foam 
loss. 

6.2.4.1.2 PDL Testing 

Hot gas tests were performed for STS-70 on multiple closely-spaced PDL 1034 repairs. For ET- 
124, two TPS repair tests were performed to re-qualify PDL 1034 supplied by a new vendor. 
These were subjected to thermal vacuum and hot gas conditions. 

A process demonstration for multiple PDL 1034 pours from a single syringe load was conducted 
at KSC and documented in TPSB SS20-613 and TPSB SS20-614. The revised process was 
implemented to increase overall efficiency of the repairs without sacrificing quality. The 
efficiency increase was desirable in order to perform repairs in a “production mode” that would 
support repair schedule milestones. The demonstration showed that three holes in close 
proximity was the limit for a one-syringe application of PDL. No dissection measurements or 
photographs were collected during these demonstrations. The process demonstration results are 
summarized in Appendix E. 

6.2.4.1.3 BX-265 Manual Spray Testing 

Key tests run to substantiate the ET-124 BX-265 sprays are outlined at Appendix C. Spray 
process demonstrations were conducted to validate the BX-265 repairs. A mockup of the ET 
nose cone “pencil sharpener” area was built-up at MAF and sprayed using processes and access 
constraints identical to those planned for the flight vehicle. The strength of the sprayed BX-265 
was measured using the plug pull technique to validate the application process. Unexpected low 
tensile strength measurements in a few plug pulls were attributed to either an off-nominal plug 
pull technique or to suspected insecticide contamination. A follow-on flat panel test spray was 
conducted and additional tensile strength measurements were taken, tests were conducted to 
determine the influence of insecticide contamination. 
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6.2.4.2 Flight History 

6.2.4.2.1 STS-96 ET-100 Hail Damage: Debris, Ice and TPS Assessment 

During on-pad processing for STS-96, ET-100 sustained over 700 instances of impact damage 
due to hail. Approximately one-third of those were acceptable “as-is”, one-third were repaired 
by blending the damage with adjacent foam, and the remaining one-third were repaired with 
PDL fills. All damaged areas in the L02 tank “no ice zone” were repaired according to certified 
design repair criteria. 

Flight rationale was presented at the STS-96 / ET-100 Pre-Launch Mission Management Team 
(MMT) Review on May 25, 1999 for flying ET-100 as-repaired. The rationale indicated the 
blended areas were returned to drawing tolerances. The sand and blended areas met all thickness 
and waviness requirements. PDL repairs were certified by a variety of tests and analyses 
including wind tunnel testing, radiant ablation tests, combined environments tests, coupon tests, 
and manufacturing validations. Tests of simulated hail damage were also conducted as reported 
in the test report ETTP-621, Hail Damage Simulation on NCFI 24-124. Stress analysis showed 
that the repaired NCFI 24- 1 24 satisfied the 1.10 factor-of-safety requirement. 



Figure 6.2-3. STS-96 ET-100 Typical Hail Damage 
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Pre-Launch Inspection 

Prior to launch of STS-96, a final pre-launch inspection was conducted. There was no Launch 
Commit Criteria (LCC), Operation and Maintenance Requirements (OMRS), or National Space 
Transportation System (NSTS)-08303 criteria violations. No ice, debris, or TPS problem reports 
were identified. 

The STS-96 Ice Inspection Team report indicated that the ET in general had no NCFI acreage 
icing concerns and no protuberance (i.e. IFR, PAL ramp, bi-pod strut) icing conditions outside of 
the established database. The TPS performed nominally during cryogenic propellant loading. 

The L02 tank acreage had visible condensate but no ice or frost formations. All hail repairs 
were intact and exhibited no thermal shorts. No anomalies were detected on the inter-tank. The 
LH2 tank acreage had condensate but no ice or frost formations. After the pad was cleared for 
launch, the Ice Inspection Team continued to monitor the vehicle with remote cameras and infra- 
red radiometers. There was no ET acreage ice observed and no noticeable increase in 
protuberance icing was detected. The report did indicate that there was extensive surveillance of 
the hail damage to verify repair integrity and the absence of ice formation. 



Figure 6.2-4. STS-96/ ET-100 Pre- and Post- Launch Comparison 
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In-Flight Assessment 

A re-configured ET intertank flown on STS-96 incorporated thousands of pin-size vent holes 
with 0.3 inch spacing on the ET -Y and +Y thrust panels. Significantly more debris was 
observed in the ET separation photography coming from the non-vented areas of the intertank 
than from the vented areas. The divots in both areas appeared to be shallow with no primed 
substrate visible. There were over 500 divots observed. 

ET/Orbiter umbilical separation films were used to observe the hail damage repairs. The repair 
areas visible in the sunlit section appeared to be intact and in good condition per the KSC report. 
At the Debris Integration Group meeting on March 28, a report from the JSC SE&I Imagery 
Integration presented results of a study of the STS-96 (ET-100) L02 tank repair performance 
after hail damage, Shuttle ascent, and ET separation [ref. 17]. The report concerned use of stereo 
image analysis to compare the pre-launch tank repairs with a post-separation photo of about the 
same region of the tank. This report observed that, post-separation, the repair areas appeared to 
have surfaces that were even with or slightly depressed compared to the surrounding foam 
surface. None of the possible depressions has measurable depth using their stereo analysis 
techniques (it was reported that depressions would have needed to be greater than about % inch 
deep to be measurable). The Imagery Integration team concluded that it is possible that one of 
the apparently depressed regions might be associated with a foam loss, but that there was no 
quantitative evidence to support the conclusion. The group reported that no other repairs 
appeared to have additional foam loss. 

A report from the Image Science and Analysis Group at JSC at the same meeting also compared 
pre-launch imagery to on-orbit umbilical camera imagery to determine whether hail damage 
repairs were observed to be lost during ascent [ref. 2], Sizes and gray-scale (brightness) 
comparisons were made between the pre-launch and on-orbit imagery. The report indicated no 
indications of change in the repair footprints, and the color of the repair areas in the on-orbit 
photos is reported to be significantly darker than newly exposed foam. The report concluded that 
there were no “significant losses” of foam repairs in the observed region of the tank.” 

An additional photographic analysis was done by Dr. William Kaukler of the University of 
Alabama in Huntsville; performed stereographic analysis of pre-launch and on-orbit photographs 
(Figure 6.2-3). Dr. Kaukler investigated 17 regions that he believed to show some indication of 
foam recession after ascent. Only one of these regions correlated with a PDL repair (the others 
presumably having been sanded regions). Dr. Kaukler reported a “negative correlation” between 
the PDL repairs and his observation of foam recession” [ref. 12], 
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Post Landing Debris Assessment 

Post-landing debris impact assessment of the Orbiter indicated 1 60 lower surface hits of which 
66 were greater than 1 inch. Boeing’s post-landing report for STS-96 [ref. 16], noted that most 
of the damage was concentrated from the nose gear to the main landing gear wheel wells on both 
left and right chines as shown in Figure 6.2-5. 



Figure 6.2-5. Lower Surface Orbiter Damage 


The Boeing report also noted the outboard damage sites on the chine areas followed a similar 
pattern documented on other missions [ref. 16], as shown in Table 6.2-2. A higher than average 

NESC Request No.: NESC 07-005-E 


# 

NASA Engineering and Safety Center 
Technical Report 

Document #: 

RP-07-47 

Version: 

1.0 

Title: 

STS-117 Hail Damage Repair Assessment 

Page #: 

23 of 113 


number of debris impacts along the chine areas was observed for STS-96. The number of impact 
damage sites greater than 1 inch was also higher than average, however, the depth of the damage 
was relatively shallow. 


Table 6.2-2. Data from Boeing Report 



Lower Surface 
(total hits) 

Lower Surface 
(hits > 1 inch) 

Longest damage 
site (inches) 

Deepest damage 
site (inches) 

STS-86 

100 

27 

7 

0.4 

STS-87 

244 

109 

15 

1.5 

STS-89 

95 

38 

2.8 

0.2 

STS-90 

76 

11 

3 

0.25 1 

STS-91 

145 

45 

3 

0.5 

STS-95 

139 

42 

4 

0.4 

STS-88 

80 

21 

4.5 

0.5 

STS-96 

160 

66 

4 

0.5 


This period in the SSP included significant changes in the ET. The Super Lightweight Tank 
(SLWT) was introduced with ET-96 concurrent with an increase in “popcoming” foam losses 
from NCFI on the intertank. An effort was made to control this through modifications to the 
surface of the NCFI foam which ultimately led to the vented configuration adopted for STS- 101 
(ET-102). ET-100 was one of the tanks in a transitional series which incorporated design 
modifications to reduce these intertank foam losses and had modified venting applied to a limited 
area as a flight test of this technique. The number of larger than one inch diameter damage sites 
on the Orbiter was higher for STS-96 than for other missions with a similar intertank foam 
configuration; however, higher-than-normal “popcoming” foam losses were observed for ET- 
100. Data in Figures 6.2-5 and 6.2-6 are from [ref. 28]. This data shows “popcorn” foam loss 
counts from flight image analysis for ET thrust panels for STS-93, 96, 103, and 101. Higher 
foam loss counts are seen for ET-100. 

While not conclusive, flight data analysis suggests the damage to the orbiter seen on STS- 100 
was caused by popcorn foam from the ET intertank and not loss of hail damage repairs. 
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Figure 6.2-6. Foam Loss Comparisons for +Y Intertank Thrust Panels 



Figure 6.2-7. Foam Loss Comparisons for +Y Intertank Thrust Panels 
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6.2A.2.2 STS-70 ET-71 Woodpecker Damage: Debris, Ice and TPS Assessment 

STS-70 (ET-71) was being prepared for launch in June 1995 when woodpecker damage to the 
ET occurred. This tank did not have the NCFI 24-124 intertank foam. The repair process on 
damaged areas included trim out and PDL repairs. More than 1 90 damage sites were reported 
and 1 67 were repaired with PDL. The remaining damage was “used as is” or sanded and 
blended [ref. 3]. Repairs began on June 8 and were completed by June 14, 1995. 

Pre-Launch Inspection 

During the pre-flight final inspection there were no LCC, OMRS, or NSTS-08303 criteria 
violations. There were no interim problem reports generated. The inspection team specifically 
checked the woodpecker damage repairs. No ice or frost formations, debonds, or material 
protrusions were observed. 

The ET in general had no acreage or protuberance (i.e. Ice/Frost Ramp (IFR), PAL ramp, bipod 
strut) icing conditions outside of the established database. The L02 tank acreage had 
condensate, but no ice or frost formations. The woodpecker damage repairs were intact and 
exhibited no thermal shorts. No acreage anomalies were detected in the intertank. There were 
typical ice and frost accumulations on the intertank Ground Umbilical Carrier Plate (GUCP). 

The LH2 tank acreage had condensate, but no ice or frost formations. 

There was no further mention in the report of the time span from the final inspection through 
launch. It is assumed there was no additional icing observed from the remote cameras, since the 
inspection team summary indicated no interim problem reports were generated 

In-Flight Assessment 

On-orbit film and video review by KSC indicated the LH2 and L02 tank acreage was in good 
condition with no visible divots or abnormal regression. No divots or TPS anomalies were 
observed at the woodpecker damage repair locations. The report also stated there was no 
indication of intertank acreage divots. The JSC Photographic Analysis Summary concurred the 
woodpecker repairs appeared to be intact. 

Post Landing Debris Assessment 

Boeing’s post-landing debris assessment indicated lower than average tile damage. There were 
81 total lower surface hits with only 5 were greater than 1 inch. The damage sites appeared to be 
randomly distributed on the aft half of the Orbiter lower surface. There was some concentration 
around the LH2 ET umbilical door. However, the uniform distribution of damage sites in the 
chine areas, seen on STS-96, was not present on STS-70 

6.2.4.23 STS-91 ET-96 Stress Relief Grove 

The ET flown on STS-91 was the first of three tanks flown with a stress relief groove cut in the 
L02 ogive acreage foam as shown in Figure 6.2-7. The stress relief groove was required to 
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relieve NCFI foam induced compression loading on a L02 ogive weld joint. The groove was 
approximately 14 inches wide, by 6 feet length, 1 inch deep; it was created by sanding and 
blending the NCFI. While the modification created significant surface waviness, flight photos 
showed no evidence of abnormal erosion or significant foam loss from the L02 tank (Figure 6.2- 
8 ). 



Figure 6.2-8. STS-91 L02 Tank Stress Relief Groove 
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Figure 6.2-9. STS-91 Post-Separation Photo 


6.2.4.2.4 Performance History of PDL-1034 Repairs 

Standard PDL-1034 repairs are test-demonstrated and have been used extensively over the life of 
the Program with outstanding performance [ref. 24], As discussed in the flight histories of STS- 
96 and STS-70, no performance issues were observed with PDL repairs used for previous hail- 
and woodpecker-damaged L02 tanks. There have been no conclusive observations of PDL 
repair losses from any L02 tank, although PDL repair losses from the LH2 tank and intertank 
have been observed and were one subject of the STS- 1 14 ET IF A investigation [ref. 15]. 

A number of documents were reviewed by NESC relevant to the historical performance of PDL 
repairs and are summarized in Table B-l of Appendix B. The PDL Verification and Validation 
(V&V) data presented in these reports is limited to four sets of data gathered using similar, but 
non-identical, processes. Pre-STS-107 V&V data did not record voids less than 0.5 inch 
(reporting size limit). The post-STS-107 longeron repair data set consisted of five samples, three 
of which had slot defects up to 0.4 inches in diameter. The last two data sets were taken from 
on-tank TPS dissections and recorded cylinder defects from 0.7 inch to 0.98 inch in length and 
slot defects up to 0.58 inch in diameter. 
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6.2.4.3 Probabilistic Risk Assessment 

The large number of repairs on ET-124 represents an unquantified increase in flight risk. In an 
effort to estimate the risk incurred by the repairs, the Shuttle Program Systems Engineering and 
Integration Office (SE&I) is performing a probabilistic risk assessment for the PDL repairs on 
ET-124 [refs. 20, 21]. This analysis assumes that the mass of a liberated repair would be equal 
to the mass of PDL represented by the repair volume. Repairs greater than the deterministic 
limit of 0.004 lbm are considered for further analysis if they were within the <p = ±1 10 degree +Z 
region of the tank. The conditional per-release hazard for single debris liberation is then 
calculated using three assumed timing distributions (uniform 35-135 seconds, following the 
heating rate, following the dynamic pressure) and the same debris transport model employed for 
previous flights. A release rate of 1/500 is assumed to arrive at a mission risk. This rate is based 
on the STS-121 (ET-1 19) flight during which one PDL repair loss was observed out of about 500 
repairs on the tank and assumed typical of most missions. There is uncertainty in this estimate: 
ET separation imagery is not captured on all flights, and not all repairs are visible in the images 
that do exist. The overall flight risk from PDL repairs is sensitive to the choice of release rate; 
for example, if the release rate is halved the risk doubles. Initial SE&I analysis using the 1/500 
rate produced an estimated maximum 1:4000 risk for repair loss from the L02 ogive [ref. 21]. 
Analysis performed using a 1/100 rate produced an estimated maximum 1:800 risk. 

6.2.5 NESC Participation in ET Project Assessment and Planning Meetings 

The NESC Team actively participated in the ET-124 hail damage repair effort from early 
evaluation of the impact to completion of the repairs. Team members participated in project 
planning and review meetings and maintained informal communication with ET Project 
management and KSC, MSFC, JSC, and LMSSC personnel. 

The hail damage daily engineering assessments were critical planning and reporting meetings 
held early in the process of recovery from the hail event. Members of the NESC Assessment 
Team participated in these daily teleconferences. The ET Chief Engineer Review Board (CERB) 
meetings were held frequently and were the forum for technical discussions of the damage 
assessment and repair planning. Those teleconferences were used to discuss repair approaches 
and their validation. NESC team members participated in these meetings, especially when key 
decision points were being proposed. The NESC team was represented in numerous Technical 
Interchange Meetings (TIMs) scheduled when a significant milestone or decision point was 
being approached and a significant amount of data was available for review. NESC Assessment 
Team members participated in the weekly Shuttle Program Requirements Control Board (PRCB) 
meetings during which the ET project routinely discussed status and progress in the ET-124 
recovery effort. Additionally, the NESC Assessment Team members engaged in numerous 
informal meetings with ET project representatives and with ET and Shuttle Propulsion Chief and 
Deputy Chief Engineers. The NESC Assessment Team Lead also participated as a member of an 
Independent Review Team (IRT) chartered by the ET Project to review and comment on damage 
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repair efforts prior to rollout from the Vehicle Assembly Building (VAB). Lockheed-Martin 
(LM) responses to actions levied by the IRT. 


7.0 Data Analysis 

Appendix C contains a list of tests, analyses and miscellaneous documents relevant to the ET- 
124 repair that were reviewed by NESC. 

7.1 Inspection and Disposition of Damage 

Damage inspections were performed by teams of experienced inspectors from engineering and 
S&MA and included both MAF and KSC personnel. Despite the extent and thoroughness of 
those inspections, the potential remains that some crushed foam may have escaped detection. 
Data collected during the STS-1 14 IF A investigation indicated that crushing up to 15 percent of 
thickness cannot be reliably detected by visual inspection. Since red dye was not applied to all 
defects as a visual indicator, especially foam in areas where damage was superficial and defects 
were dispositioned for use as-is or where sand-and-blend repairs were performed (see section 
6.2.3. 1), the possibility that some crushed foam remains is increased. 

The potential for undetected damage to the tank shell is minimal. Any hail impact sufficient to 
cause structural damage to the tank would have left visible damage on foam as well. No pressure 
vessel substrate was exposed by any of the impacts. 

Inspection of composite components (nose cone, gaseous hydrogen (GH2) pressurization line 
fairing, and intertank access door) revealed no anomalies. NESC reviewed and concurs with the 
inspection assessment (Appendix J). 


7.2 Crushed Foam 

The body of crushed foam test data produced in support of the STS-1 14 IF A is extensive, and is 
supplemented by hail damage simulation test data gathered during STS-96. Panels with foam 
crushed at various levels were subjected to thermal-vacuum and hot gas testing during the IFA 
work and liberation of debris with a mass in excess of the deterministic limit (0.004 lbm) 
observed only when large-diameter (3-4 inches) indenters were used to crush the foam to 
approximately 20 percent. Crushed foam debris liberated during hot gas testing was observed to 
fragment into small pieces immediately on release [refs. 13, 19]. Foam at 51 damage sites on 
ET-124 was treated with red dye and removed incrementally to assess damage depth. L02 NCFI 
crushing did not extend beyond about 0.2 inch of the defect bottom, and ET intertank crushing 
was limited to a depth of about 0.1 inch of the defect [ref. 1]. Data from dissection of STS-96 test 
panels is consistent with these observations; however, since the STS-96 simulations do not 
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envelope the STS-1 17 hail event due to significant differences in magnitude, rate, and cross- 
sectional area over which energy was dispersed in those test articles, NESC recommended 
additional tests be conducted on panels designed to simulate the STS-1 17 damage. These tests 
showed a linear relationship between energy of impact and depth of TPS damage consistent with 
the STS-96 results, with the bulk of the crushed foam found within about 0.2 inch of the defect 
bottom [refs. 5, 24], Hot gas tests were conducted on un-repaired NCFI panels subjected to STS- 
117 simulated damage (both static and dynamic) and no significant debris release was observed 
[ref. 8]. Recession rates for these panels were in-family. 



Figure 7.2-1. Hot Gas Test of NCFI Panel Subjected to Static Loads Simulating STS-117 

[ref. 8] 

On the basis of these tests, NESC concurs that significant debris release or accelerated TPS 
recession due to residual crushed foam is unlikely. 


Surface and Interstitial Ice 

The potential for ice to form on or in areas of residual crushed foam was assessed. Results of 
multiple cyrogenic cycle and thermal vacuum tests performed on an NCFI panel with “barely 
detectible” crushed foam (crush level of approximately 1 5 percent) were reviewed. No ice 
formed during any of the 9-hour cryogenic cycles at average ambient temperatures of 62 deg F 
and 93 percent relative humidity. No debris in excess of the deterministic level was liberated in 
the subsequent thermal -vacuum test [ref. 6]. The test setup and conditions were conservative and 
bracket conditions expected at KSC for a June or July launch. 

If the outer surface of the NCFI is at 90 deg F and the inner surface is at -297 deg F (L02 
cryogenic temperature), the temperature will be 32 deg F, 20 percent of the way into the foam 
thickness. That is, for 1 inch of foam, the 32 deg F isotherm will be 0.20 inch below the surface. 
The minimum thickness of foam remaining in the sand and blend repairs is 0.65 inch, so freezing 
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temperatures at those locations can occur 0. 1 3 inch below the surface. If ambient temperatures 
are lower, sub-freezing temperatures can be expected even closer to (or the same as) the NCFI 
surface temperature. In the test series referenced earlier, temperatures as low as 30 deg F were 
detected at the surface 6 hours into cryogenic chilldown. The implication is that sub-freezing 
temperatures can be expected within a region of undetected crushed foam regardless of ambient 
temperature, so the possibility that ice may form in a void created by damaged cell walls internal 
to a region of crushed foam (interstitial ice) cannot be dismissed on the basis of temperature 
alone. 

Data collected during development of NESC hydrophobic coatings offers some insight into the 
mass of water that can be expected to accumulate in undamaged NCFI (Appendix H). Absent a 
communication path from the foam interior to the surface, crushed NCFI can be expected to 
contain a similar amount of liquid water. One inch core samples taken from freshly sprayed and 
1 5-month weathered NCFI panels contained a maximum of 0.48 to 1 .48 grams of water, 
respectively, within 0.25 inch of the NCFI surface (Appendix H, pg 8-9). Hail-damaged crush 
sites seen on ET-124 were on the order of an inch in diameter and, as noted previously, had 
damage typically extending no more than 0.2 inch beneath the surface. The amount of water 
contained in the 1 inch x 0.25 inch samples tested thus offers a reasonable estimate of the release 
mass that can be expected if all the water in a damage site was frozen and liberated as a single 
piece of debris. 

Water accumulation significantly greater than the values above is unlikely, even if a 
communication path from the surface to the interior of a crushed foam region does exist. Air 
trapped in the cavity will prevent entry of liquid water unless a vent path is also present, and 
surface tension of the liquid will inhibit flow. If liquid water does find its way into a cavity from 
the NCFI surface, the increased thermal conductivity would tend to warm the deeper layers and 
reduce the likelihood ice will form. Cavity volume would be limited by the size of the damaged 
area. Again, samples collected from ET-124 and in conjunction with STS-96 and STS-1 17 
damage simulations suggest crushed areas on the order of an inch in diameter by no more than 
0.25 inch deep are to be expected. The presence of undetected large-scale crushed areas in the 
ET-124 tank acreage is unlikely, given the extensive inspections performed after the hail event. 

Two mechanisms exist that could precipitate ice debris liberation from a region of crushed foam: 
void-delta pressure (V-dP) and aerodynamic heating / erosion. Neither provides a likely 
mechanism for liberation of surface or interstitial ice in crushed foam, even if such ice were to 
form. The V-dP mechanism would require presence of a void in the acreage foam adjacent to the 
ice, formed either during application of machine-sprayed NCFI or as cell walls were collapsed 
during crushing of the foam. Machine-sprayed NCFI has a low incidence of void entrapment, 
and the probability that multiple, disconnected voids would form in crushed foam oriented in 
such a way that one would produce ice while the other did not is remote. Aerodynamic heating / 
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erosion would ablate the material from outside in and extensive hot gas testing indicates it does 
not typically produce debris in excess of the 0.004 lbm deterministic limit. 

At this time, ambient temperature conditions on day of launch are expected to be much higher 
than the conditions successfully test demonstrated for known crushed foam conditions. Rain 
would increase heat transfer to the foam surface that would decrease the likelihood of icing. In 
addition, multiple day of launch inspections using visual, IR and SURFICE techniques 
are planned to ensure to acceptable launch conditions. 

Given the above rationale, NESC concurs that surface or interstitial ice is unlikely to form in the 
vicinity of any residual crushed foam on ET-124 and the debris risk posed by such is remote. 

7.3 Repair Processes 

Standard repair processes appear adequate with procedures providing sufficient level of detail to 
minimize the potential for human errors. All repair processes applied to ET-124 have been 
validated by test. Technicians performing repairs were trained and certified to perform the work. 
The Kennedy Space Center (KSC) workforce received safety classroom training and on-the-job 
task-specific training. ET foam repair certifications (Critical Skill Certification CSR 889, “ET 
Foam Application for Repair”) did not include dissection and analysis of PDL sample pours. 
Only experienced MAF technicians performed non-standard repairs. 

Red dye was used as a damage indicator during inspections and in conjunction with all the repair 
procedures employed on the tank. Effectiveness of the dye as an indicator was demonstrated by 
test during the STS-1 14 IF A investigation. 

7.3.1 Sand and Blend Repairs 

In an attempt to minimize the removal of undamaged NCFI from ET-124 and also limit the 
number of PDL repairs required, minimum foam thickness criteria were reassessed and new sand 
and blend contours were developed and verified during the hail damage recovery effort. Wall 
angles of the blends are steeper than those that would have resulted had the original 
aerodynamically-derived blending requirements required by 9010 been applied and many of the 
blends are deeper than the drawing-specified minimum foam thickness. NESC reviewed the test 
and analysis that provides the rationale for revisions to the drawing requirements and 9010 
criteria. 

7.3.1. 1 NESC Analysis of Waviness Criteria 

During the development of the Space Shuttle, an ET outer mold line surface waviness criteria 
was defined by the Shuttle thermal community so that the manufacturing team would have a 
reasonable guideline as to how smooth the ET foam surface needed to be. The original spray-on- 
foam insulation (SOFI) equipment produced a corrugated, spiral pattern that would increase the 
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local aerodynamic heating, at the top of the corrugations, relative to an “undisturbed” flat plate 
value that was determined from smooth wind tunnel models. The original waviness criterion 
was sized so that the local heating, on the L02 tank ogive, would not exceed 1 .3 times the 
corresponding smooth surface value. The corresponding “wave” amplitude and length for the 
ET ogive and intertank are specified in Martin Marietta Corporation (MMC) drawing 
80971 1 1 8408. The actual dimensions vary with position along the ogive, with the smallest 
wavelength and amplitude at Xt=371 being 3 inches and 0.18 inch, respectively. 

Unfortunately, a similar criteria for cavity dimensions has never been incorporated into the ET 
drawing system, thus any cavity type feature has historically been treated as if it was a surface 
wave even though the local flow physics of the two features is very different. The diameter of 
the cavity, at the SOFI outer surface was being used as the wavelength, and the depth of the 
cavity as the wave amplitude. Any cavity that did not meet the aerodynamic waviness criteria 
was repaired per the directions in MMC drawing 80901019010, which requires that the surface 
“smoothness” meet the aerodynamic waviness criteria. 

The large number of hail damage sites, on ET-124, brought this issue to the SSP Thermal Panel 
several times during March of 2007. During the discussions, Marshall Space Flight Center 
(MSFC) Engineering noted that cavity heating test data had been developed and compared it to 
the literature after the STS-96 hail damage event. This work was presented to the SSP Thermal 
Panel on March 15, 2007 and a recommendation made that the “aerodynamic waviness” criteria 
not be used to size the cavity repairs. Doing so would cause good NCFI foam to be removed 
unnecessarily as cavities were enlarged to meet the aerodynamic waviness criteria. It was further 
recommended that the cavity walls be sloped to minimize the local heating to the downstream 
wall. The SSP Thermal Panel and NESC concurred with these recommendations. MSFC 
Engineering and the ET Project developed and implemented a revised cavity repair specification 
for the Sand and Blend and Use As Repaired (USR) or Use As Inspected (USI) repairs. 

Supporting thermal analyses are documented in two SSP Thermal Panel presentations, one by 
Michelle Guillot of Lockheed Martin, and the other by Tibor Lok, a USA consultant and IRT-24 
demonstrated that the NCFI bond line temperature was a strong function of the internal L02 
ullage temperature and only a weak function of the external aerodynamic heating. Thus, even 
large changes to the external heat transfer rate, such as a factor of two or three, only cause 
modest increases to the maximum bond line temperature which occurs just before Space Shuttle 
Main Engine (SSME) shut down, well after peak aerodynamic heating. Panels configured with 
representative rounded-edge and 0.5 inch sanded- wall-angle sand and blend repairs were 
subjected to hot gas testing on April 14, 2007 [ref. 7]. Results demonstrated acceptable recession 
rates and no significant debris liberation. 

NESC concurs, on the basis of these tests and analyses, that the modified waviness criteria 
developed for ET-124 are acceptable for flight and the shape of the sand and blend repairs should 
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be optimized to minimize ice formation prior to launch, instead of only minimizing ascent 
heating. 



Figure 7.3-1. Hot Gas Test of NCFI Panel Containing Multiple Steep- Wall Sand and 

Blend Repairs [ref. 7] 


7.3.1.2 NESC Modified Sand and Blend Criteria and Minimum Foam Thickness 

ET Project developed an option to perform deeper than normal sand and blend (S&B) repairs, 
i.e., repairs that would violate the minimum NCFI foam thickness required by the external tank 
drawings. The minimum NCFI thickness required to prevent external ice formation on the 
oxygen tank while ensuring an 85 percent launch probability between May and October was first 
established. The Program’s 30-year KSC weather data base was used as input to an ice 
simulation program and the NCFI thicknesses in the no-ice and limited icing zones were adjusted 
until the 85 percent criterion was met. This effort resulted in minimum foam thicknesses of 
0.8 inch in the no-ice zone and 0.6 inch in the limited ice zone vs. the established drawing 
minima of 1 .0 inch. 

The second step in this effort was to devise a method of sorting the hail damage locations into 
S&B and PDL repair. That is, to identity the icing potential of each prospective S&B repair and 
to repair with PDL those that might ice. To enable this sorting, ET Project devised an icing test 
in a natural convection chamber. A box fan was included in the test to allow for some forced 
convection. Liquid nitrogen-backed foamed panels with cavities of different depths and shapes 
were tested to identity the icing limits. The results were used to sort the repairs into PDL and 
S&B, resulting in 26 suggested PDL repairs and 301 S&B repairs. Three repairs that did not 
meet the sort criteria were re-designated as PDL due to concerns about an increased icing risk at 
these locations and depths. 
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Figure 7.3-2. Sand and Blend Criteria Icing Test [ref. 10] 


NESC investigation of the historic weather at KSC showed that the 85 percent cutoff coincided 
with wind speeds that would cause the heat transfer on the L02 ogive to be dominated by forced 
convection; as a consequence, there was an issue with directly using the results of a natural 
convection test with limited forced convection to assess the potential for S&B icing. NESC 
performed a physics-based non-dimensionalization of the icing potential to allow a relative 
assessment of the S&B icing potential from all the repair sites (Appendix F). The analysis 
showed that the three repair sites that had been re-designated for PDL did indeed have icing 
potential that exceeded other S&B sites, thus confirming the ET Project selection. The analysis 
also identified 1 1 other repair sites that had been selected for S&B that had icing potential which 
exceeded that of sites that had already been designated for PDL repair. NESC recommended 
these sites be repaired with PDL, and ET Project accepted the recommendation. Because the 
tests performed to establish; the PDL versus S&B repair criteria were not fully representative of 
the launch pad heat transfer physics, the possibility remained that launch probability due to icing 
could be substantially less than 85 percent. To address this concern, NESC performed a 
sensitivity analysis using the 30-year KSC weather set and demonstrated that, even if the physics 
used to set the cutoff was off by a factor of two, the probability that day-of-launch icing would 
not exceed Launch Commit Criteria (LCC) requirements still exceeded 55 percent. This 
assessment is detailed in (Appendix G). 
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7.3.2 PDL Repairs 

Over 900 PDL repairs were made on the ET L02 tank, which is approximately an order of 
magnitude more than normal. In general, the specified PDL repair density limits (repairs per 
square foot and distance between repairs) have been test-validated and demonstrated to have no 
effect on NCFI integrity. 



Figure 7.3-3. STS-70 Multiple PDL Repair Test Panel [ref. 3] 


PDL material used on ET-124 had to be requalified due to a vendor change. Testing done to 
support the requalification also demonstrates acceptable performance of multiple PDL and S&B 
repairs in an environment similar to that expected on the L02 tank at ascent [ref. 5]. 
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Figure 7.3-4. Hot Gas Test of PDL Repair Panel Containing Multiple Repairs [ref. 3] 


7.3.2.1 NESC Human Factors Assessment of PDL Application Processes 

The NESC Human Factors (HF) team reviewed procedures and working conditions prior to 
execution of PDL repairs on ET-124. The team visited the VAB work area, observed a 
damaged-foam removal demonstration and a procedure development “table top” review, and 
interviewed engineers and technicians involved in the work. The HF team did not make any 
direct observations of repairs in-progress on the tank or on high-fidelity mockups. 

Procedures incorporated relevant input from all major stakeholders including technicians, 
engineering, and Quality Control (QC) representatives from the United Space Alliance (USA), 
NASA Engineering from KSC and MSFC, and Lockheed Martin, MAF. Work steps included 
appropriate inspection points and were written in accordance with standing procedures, 
requirements, and standards. The PDL repair processes varied slightly from those implemented 
in the past at MAF and KSC. In order to complete the number of repairs required in a reasonable 
amount of time, KSC adopted an “assembly line” process wherein defects were addressed in 
parallel instead of individually; i.e., rather than fully repair one defect at a time, technicians 
performed surface preparation of all defects, applied Conathane® to all defects, installed molds, 
poured PDL and allowed it to cure, then removed all the molds and performed final trimming. In 
another change from the normal process, a single charge of uncured PDL was used to fill as 
many as three separate defects. These changes allowed for significant gains in processing 
efficiency. The three pour procedure was successfully demonstrated prior to implementation. 

The NESC did note that while a maximum allowable delay before pouring PDL at a given 
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temperature is specified, the procedures did not include verification steps to ensure this pour time 
was not exceeded and recommended quality inspector timing of multiple PDL pours from a 
single syringe to the procedure development team. The team also recommended that red dye 
application procedures be strengthened with specific work steps in lieu of notes. These 
recommendations were subsequently implemented. 



Figure 7.3-5. ET-124 Access and Lighting in the VAB 
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Figure 7.3-6. ET-124 Nose Cone Area Access 

Training, work area access, and lighting were reviewed. The PDL repair certification / re- 
certification procedures used at KSC and MAF are slightly different. Most notably, the KSC 
training procedures do not include steps for dissection and analysis of PDL repairs, and the test 
panels used for KSC training are not made with NCFI foam. While access to the work area was 
adequate, NESC noted the lighting in the LOX ogive work area was insufficient for some tasks 
such as inspection of Conathane® application and identification of damaged foam prior to red 
dye application. NESC recommended improvements to lighting in the work areas, and that 
recommendation was accepted and implemented. 

The Project was diligent in ensuring all known critical parameters were controlled. PDL repairs 
were not subjected to detailed evaluation to ascertain what effects minor process variations may 
have on internal void size or distribution. The existing body of PDL repair void data is 
summarized in Appendix B, Table B-l and assumed applicable to the repairs done on ET-124, 
though no recorded data exists to validate this assumption. While the bulk of the repairs done at 
MAF are performed with the tank horizontal, all ET-124 repairs were done with the tank vertical 
As the data in table B-l indicates, this increases the potential for void formation. Process repair 
demonstrations were conducted on BX-265 panels to validate the three pour / one syringe 
technique, but the repair samples were not subjected to controlled dissection and no void size or 
distribution data was recorded documented in KSC TPSB SS20-613 and TPSB SS20-614, 
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(Appendix E). Approximately 32 repairs were removed from the tank to address a concern 
relating to the underlying Conathane® application but again, no sub-surface void data was 
collected as the repairs were removed. The on-tank dataset is thus limited to verbal “no voids 
noted” reports gathered in conjunction with the validation tests and repair removals. 

Despite a variety of process controls, the sheer number of repairs increased the probability that 
process escapes or creep could occur. Manually sprayed foam applications are normally 
validated by lead-in / lead-out tests done on mock-ups, and NESC recommended this process be 
extended to the hail damage repairs through routine collection of sample data as those repairs 
were performed. Ideally, sample repairs would have been made on NCFI or BX-265 panels off 
the tank at the beginning and end of every processing shift and those repairs dissected to monitor 
sub-surface void size and distribution. This would have yielded a body of data to validate the 
void distributions assumed for the subsequent PRA and would have provided visibility to ensure 
no process creep was occurring as the repairs were completed. The Shuttle Program did not 
implement this recommendation, and no in-process data was collected. The Program proposed 
gathering additional data from PDL test pours after all on-tank PDL repairs were complete. Such 
data would not have been sufficient to address the process creep concern, however, and given the 
unknown effects minor process variations may have on internal void size and distribution may 
have led to erroneous conclusions. Consequently, NESC concurred with not gathering PDL test 
pour data after the on-tank PDL repairs were complete. 



Figure 7.3-7. Sectioned PDL Repair Following Hot Gas Test [ref. 5] 
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Although no on-tank void distribution data was collected, it is reasonable to assume the quality 
of the ET-124 repairs does not differ significantly from those applied to other tanks, with most 
repairs similar to the one depicted in figure 7.3-7. PDL repair application is a multi-step process, 
and as noted earlier the Project was diligent in ensuring that controls were placed on each step in 
the process. Previous dissection data indicates that internal voids are expected in PDL 
applications (see Section 6.2.4.2.4). The bulk of the repairs are small (approximately linch in 
diameter by 0.5 inch deep) and any extremely large voids would likely have breached the repair 
surface and failed inspection. NESC concurs with the use of previously-collected MAF data for 
the PRA with the recognition that lack of specific knowledge regarding the size and location of 
sub-surface voids in ET-124 repairs adds additional uncertainty to the results. 

7. 3 . 2.2 PDL Repair Process Control Issues 

The current Shuttle PRACA definitions of process escapes and catches are [ref. 18]: 

• “A process escape is defined as any problem identified after it should have been detected 
during normal processing. Process escapes include problems found during surveillance 
sampling, inspection (including random), or audit after final closeout, or final flight 
configuration verification.” 

• “A problem can be defined as a process catch if it is identified during normal processing 
(departing from procedure), inspection, or surveillance sampling prior to final closeout or 
during testing.” 

Several process control issues (catches and escapes) occurred during ET-124 repairs, including a 
Conathane® application issue, collateral damage during mold removals, and a PDL maximum 
hardness verification issue. These process issues provided an indicator that existing process 
controls were not completely effective, and they raised questions about additional process issues 
that may have occurred but were not identified as either process catches or escapes. 

The PDL repair rework rate resulting from these process control issues exceeded 30 percent (32 
re-repairs due to the Conathane® application issue, 88 missed Shore A hardness tests, 
approximately 300 sand and blend repairs due to collateral damage incurred during mold 
removals, and several post-application repair discrepancies). Rework rates from comparable 
industry processes can be expected to be at least an order of magnitude lower than the PDL 
repair rework rate. For comparison, six-sigma process performance corresponds to 3.4 defects 
per million units, or a defect rate of 0.00034 percent. In statistical process control, the process 
boundaries used to support calculations of process capability (C p indices) usually do not include 
post-process inspections, so process catches and process escapes do not affect measures of 
process capability. 
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Conathane® Application Process Issue 

KSC OP-300 Step 16 states: “Conathane® is to be applied to a thickness of 0.003 to 0.010 
inches, measured with 1 each wet film gage.” However, a note in the procedure also stated: 
“Adhesive thickness shall be determined using a wet film gage, when necessary. In the event 
that an area is inaccessible to these instruments, a visual verification of thin, uniform coating 
shall satisfy the thickness requirement.” The wet film gage is required for all thickness 
verifications. The procedural discrepancy with the requirement was identified by USA Quality 
Control, and the corrective action was a permanent deviation (redline change) to the operating 
procedure that removed the visual verification option. Approximately 32 discrepant repairs were 
removed and those ET locations were re-repaired. 


Post- Application Discrepancies 

Several discrepancies were noted during post-application inspection of repairs, including some 
PDL underfills and surface voids. The discrepant repairs were removed from the tank and 
replaced. 

Collateral Damage During Mold Removal 

Minor unexpected damage to surrounding NCFI occurred when PDL pour molds were removed. 
The “production line” repair environment left molds installed on the tank for a longer period of 
time than is typical when individual repairs are made. As a consequence, the sealant used to 
attach the molds lifted some NCFI rind as it was removed. The damage was removed by light 
sanding. 

PDL Maximum Hardness Verification Process Issues 

MAF work instructions contain maximum hardness verification for all PDL repairs per 
acceptance criteria. Shore A hardness tests are performed to ensure that the hard surfaces of the 
PDL repairs have been sanded/removed. This test was typically not performed on sanded foam 
at KSC during OP-300. Lockheed Martin Engineering identified the omission as a process issue 
following an in-depth comparison of KSC and MAF procedures, and the paperwork was changed 
to have Shore A hardness tests performed on all PDL applications on the L02 tank. Hardness 
tests were also to be performed for accessible PDL repairs on the LH2 tank. A Material Review 
(MR) action was initiated for all inaccessible locations on the LH2 tank. During post-test paper 
reviews, it was discovered that 88 PDL applications from the initial group of L02 tank repairs 
did not have Shore A hardness tests performed. KSC Engineering wrote an operation to perform 
the hardness test on the initial 88 repairs, to date this work has not been completed and the 
Project is working towards acceptance of a "use as is" MR. 
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7.3.2.3 PDL Process Improvements 

ET-124 repairs benefited from lessons learned during the STS-1 14 IF A investigation. Multiple 
red dye applications were made to ensure all crushed foam was removed from PDL repair 
locations, thus minimizing the potential for entrapment of a void in crushed foam beneath a 
repair. ET-124 repairs were of a relatively simple geometry, with NCFI excavated from the tank 
using Dotco® cutters similar to those pictured in Figure 7.3-8. These cutters excavate a circular 
hole with a square-edged bottom identical to the cutter profile. The NESC Human Factors team 
noted that application of a radius to the bottom shoulder of the cutting blades would produce a 
bathtub shaped hole less likely to trap voids during PDL backfill. A Dotco® tool guide 
developed by technicians for use as a shop aid in excavating NCFI foam illustrates a creative 
process improvement to increase repair reliability. Unfortunately, the Dotco® tool guide was not 
fielded in time to support the ET-124 repairs. Other such improvements could be implemented 
to minimize the size and number of voids produced by the repair process. 



Figure 7.3-8. Typical Dotco Cutter Profiles [ref. 22] 

7.3.2.4 PDL Repair Summary 

PDL repairs have been routinely implemented throughout the life of the Shuttle Program and are 
supported by test and analysis. Flight history, though necessarily limited due to tank visibility 
and photographic resolution issues, has shown few repair losses over the course of the Program; 
indeed, no losses from the LOX tank have been positively identified. PDL application 


2 Is a registered trademark of Dotco, Inc. Corporation Ohio Hicksville Ohio 
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procedures developed at KSC appeared adequate and all steps in the repair process were 
protected by process controls. Although the void distribution of on-tank repairs is assumed 
identical to that observed in repairs done at MAF, no data exists to substantiate this assumption 
and no in-process data was collected as a control against process creep. Such data would 
strengthen flight rationale, but NESC concurs with flight on the strength of the process controls 
known to be in place when the repairs were implemented. 

7.3.3 BX-265 Repair Sprays 



Figure 7.3-9. Partial View of ET-124 Pencil Sharpener BX-265 
Spray Repair Area [ref. 24] 


The BX-265 spray process was validated with a demonstration spray performed on an ET nose 
cone mockup [ref. 11]. Post-test dissections revealed no voids. NESC reviewed the plug-pull 
data collected in conjunction with the test and with a subsequent flat-panel spray and concurs 
with ET Project that the low values observed in the demonstration samples likely occurred due to 
contamination of the test article surface. Data collected from lead-in / lead-out sprays performed 
when the BX-265 repair was made to the flight article were well within spec and adequate to 
demonstrate acceptable material properties. The sprays were applied to the L02 tank by 
experienced MAF technicians using identical techniques and equipment. Adhesion of the BX- 
265 material to the Conathane® layer has been demonstrated in previous tests conducted to 
validate the PAL ramp repair process and confirmed through hot gas tests performed for ET-124 
[ref. 11]. These tests demonstrated BX-265-over-NCFI performance in environments similar to 
those expected in the ET-124 L02 ogive area on ascent. No foam liberation in excess of the 
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0.004 lbm deterministic limit was noted, and no significant erosion occurred. This testing 
envelopes conditions expected in the aft portion of the L02 tank. NESC reviewed the thermal 
analysis performed by ET Project to demonstrate no significant increase in bondline temperature 
would occur at the BX-265 to NCFI interface and concurs with the assertion that the 300 degrees 
F temperature limit will not be violated [ref. 24], 

BX265 and NCFI Coefficient of Thermal Expansion (CTE) Mismatch 

At -320 deg F (L02 substrate temperatures), aluminum has a CTE of -0.0031, NCFI 24-124 a 
CTE of -0.0166, and BX-265 a CTE of -0.01812 [ref. 14]. The difference of about 10 percent 
between BX-265 and NCFI is not sufficient to impart stress in the TPS beyond the limits of 
material capability and does not pose a concern for in-flight debris liberation. The 0.5 inch layer 
of BX-265 will not insulate the NCFI sufficiently to impart significant internal stress in the 
material, and thermal cracking of the kind seen in the PAL and ice / frost ramp area is 
improbable. The underlying structure in the ogive areas of interest is stable and will not impart 
undue loads in the TPS. 

On the basis of these tests and analyses, NESC concurs that the debris release above 
deterministic limits or accelerated TPS recession in the areas of the BX-265 spray repairs is 
unlikely. 

7.4 Flight Risk Assessment 

The Program’s PRA approach is identical to that of previous missions and suffers from the same 
limitations, primarily the uncertainties inherent in estimates of debris mass, debris release timing, 
transport, and orbiter impact damage. As noted in previous NESC assessments, the PRA should 
not be used as a discrete estimate of flight risk but is suitable only for assessing the relative risk 
posed by various debris sources. 

Two key variables affect the ET-124 PRA, the PDL repair release rate and release mass 
estimates. SE&I approached the release rate estimate by assuming 1 repair out of 500 would be 
lost, given the performance history of similar repairs on previous missions [ref. 21]. The history 
of PDL repair losses from the L02 ogive (the area of primary interest and that which poses the 
highest flight risk due to the potential for debris transport to critical locations on the orbiter) has 
been difficult to ascertain due to limitations inherent in flight imagery. Clearly, no wholesale 
loss of PDL repairs has been observed but estimates of actual repair losses over the history of the 
Shuttle Program are not definitive with verbal estimates ranging from none to two. SE&I 
assessed the sensitivity of the 1/500 estimate by doing a comparative analysis using a 1/100 
release rate. Resultant failure probabilities ranged from 1 :4000 for the lower rate to 1:800 for the 
higher [ref. 21]. NESC concurs with the approach, given the absence of anything more 
substantive on which to base the analysis; however, the resultant uncertainty is high and cannot 
be taken as a discrete estimate of flight risk. 
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SE&I approached the release mass estimate by assuming complete repairs would be lost from the 
tank and that the total mass of any given repair would thus provide a conservative estimate of the 
debris mass. Only those repairs with a mass greater than the 0.004 lbm deterministic limit and 
located in the critical debris zone (cp = ±_1 10 degree +Z region of the tank) were used for initial 
analyses. While this approach simplifies the analysis, it does not address the physics underlying 
V-dP foam losses and may not be conservative. A divot would originate at a void in the PDL 
repair. As it is released it typically tears away overlying material, resulting in a cone-shaped 
chunk. A divot shaped like the frustum of a cone would be expected to have a cone half angle of 
60 degrees, with the smaller end having the surface area of the original defect. The resultant 
mass of PDL and the NCFI in the divot can easily exceed that of the repair. When the mass of 
the PDL repair itself is used as the mass loss, there are only 86 repairs with masses >0.004 lbm. 
When the adjacent NCFI tom out in the divot is included, there are more than 700 possible losses 
with masses above 0.004 lbm. Adding the adjacent NCFI volume significantly increases the 
number of repairs which must be assessed but provides a more realistic estimate of the risk. 
NESC recommended the PRA be conducted using divot masses so-calculated rather than 
performing the initial sorting by the total repair mass initially planned. The NESC analysis 
methodology and results are at Appendix I. The Program concurred with this recommendation. 

7.5 Summary 

Tests and analyses performed to substantiate the repairs planned for ET-124 were well- 
formulated and provide an adequate foundation for flight rationale. The biggest weakness in 
flight rationale is the assumption that the sub-surface void distribution for on-tank PDL repairs is 
in family to that observed in repairs done at MAF. Although no direct evidence exists to 
substantiate this assumption, the NESC concurs with flight on the strength of the process 
controls known to be in place when the PDL repairs were implemented. 
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8.0 Findings, Observations, and Recommendations 

8.1 Findings 

F-l. Inspections and engineering assessment conducted by ET Project were rigorous and well- 
implemented; however the potential that some residual crushed foam remains on ET-124 
cannot be eliminated. 

F-2. Review of test databases indicates crushed foam tests do not fully envelope ET-124 due 
to differences in magnitude, rate, and cross-sectional area over which test samples were 
crushed. 

F-3. Debris release in excess of deterministic limits due to residual crushed foam on ET-124 is 
unlikely as is the potential for accelerated erosion. 

F-4. The additional component of risk offered by multiple repairs is difficult to quantify, but 
this risk is mitigated by established process controls. 

F-5. Modified sand and blend surface waviness and wall angle criteria are test-substantiated 
and adequate to address flight safety risks. 

F-6. Eleven sites selected for S&B repair should be resdesignated for PDL on the basis of 
icing potential as demonstrated by the NESC non-dimensionalized analysis. 

F-7. Specified repair density limits (repairs per square foot and distance between repairs) are 
test- validated with no adverse effect on NCFI integrity. 

F-8. Additional controls on PDL repairs are necessary to ensure pour time and red dye 
application requirements are not violated. 

F-9. KSC training procedures do not include steps for dissection and analysis of PDL repairs, 
and the test panels used for KSC training are not made with NCFI foam. 

F-10. Lighting in the LOX ogive work area was insufficient for some tasks. 

F-ll. The existing body of PDL repair void data is summarized in Appendix B, Table B-l and 
assumed applicable to the repairs done on ET-124, but no recorded data exists to validate 
this assumption. 

F-l 2. PDL repair processes do not provide for collection of in-process data. 
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F-13. Controls applied to the PDL repair process are adequate, but some process catches and 
escapes have been noted. The large number of repairs applied to the tank increases the 
possibility that undetected process escapes may have occurred. 

• Conathane® application issue drove removal of 32 repairs 

• Collateral damage occurred during mold removal 

• Some under-fills and surface voids were noted which required re-repair 

• Omission of post-repair Shore-A hardness testing for 88 repairs 

F-14. PDL repair processes could be improved through a detailed P-FMEA and process 
sensitivity study. 

F-15. Debris release above deterministic limits or accelerated TPS recession in the areas of the 
BX-265 spray repairs is unlikely. 

F-16. Total repair mass does not provide a conservative estimate of potential debris mass loss. 
Debris mass may exceed total repair mass when cone-shaped divot models are employed 
for mass calculations. 

8.2 Observations 

0-1. Review of repair processes and flight history highlight no areas of concern not already 
addressed. 

0-2. Primary issue facing ET-124 is the cumulative risk posed by repeated performance of 
process-sensitive repair tasks. 

0-3. Sand-and-Blend, PDL and BX-265 repairs are process-sensitive tasks and developmental 
testing and process controls are the primary debris risk mitigators. 

0-4. The PDL repair process is known to generate non-detectable sub-surface voids which can 
liberate debris in flight. 

0-5. STS-96 flight history (ET-100 hail damage), and STS-70 flight history (woodpecker 

damage) show minimal foam loss. Orbiter damage seen on the STS-96 mission coincides 
with changes made to ET intertank TPS and was probably caused by loss of “popcorn” 
form from that area of the tank. 

0-6. Few repair losses have been noted from other flight tanks, especially from the L02 tank 
region, despite the large number of repairs performed; however, flight experience is 
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based on limited field-of-view, post-sep imagery and is subject to some limitations. 

0-7. Previous repairs have been done using processes that are similar, but not identical, to 
those employed for ET-124. Chief differences include the vertical attitude of the tank 
and modifications implemented to facility “mass production” of ET-124 repairs. 

0-8. PRA is useful only as a tool for comparing relative risks from debris sources and should 
not be taken as a measure of the absolute risk. 

0-9. Increment of risk assumed in multiple PDL repairs is difficult to quantify. 

0-10. Repair release rates drive the risk assessment results but those rates used in the STS-1 17 
PRA are estimates and subject to uncertainties inherent in limited data. 

8.3 Recommendations 

R-l. Proceed with flight of ET-124 as-repaired [F.l - F.15]. 

R-2. Perform additional crushed foam testing on panels with simulated hail damage 
(implemented) [F.2], 

R-3. Perform PDL repairs instead of sand-and-blends at 1 1 locations identified as high-risk for 
icing (implemented) [F.6]. 

R-4. Collect in-process data during performance of in-place PDL repairs (not implemented) 

[F.l 1, F.l 2], 

R-5. Implement specific improvements to address concerns noted during human factors team 
review of the KSC PDL repair process (implemented) [F.8, F.10] 

• Time PDL pours 

• Document red dye application processes in specific work steps vs. procedural notes 

• Improve workplace lighting before application of PDL repairs 

R-6. Update the KSC PDL repair certification/recertification procedures to include steps for 
dissection of the PDL pours made on foam test panels. After dissection, require 
technicians to identify and measure any subsurface voids that are present [F.9]. 

R-7. Perform formal Process Failure Modes and Effects Analyses (P-FMEAs) on ET repair 
processes in order to identify and mitigate potential process escapes and process catches 
[F.13, F.14], 
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R-8. Mass estimates based on V-dP physics (cone-shaped divot with 60-degree wall angles) 
should be used for PRA mass loss estimates. 

9.0 Lessons Learned 

L-l . Performing trend analyses on process catches and process escapes will provide a more 
proactive and robust approach for identifying and fixing process control issues. The 
Space Shuttle Program (SSP) currently requires trending and reporting only process 
escapes to the SSP Quality Panel and SSP managers [ref. 18] [F.13, F.14], The SSP 
definitions of process escapes and process catches are inconsistent with the industrial and 
human factors engineering terminology used in industry. 


10.0 Definition of Terms 

Corrective Actions Changes to design processes, work instructions, workmanship practices, 
training, inspections, tests, procedures, specifications, drawings, tools, 
equipment, facilities, resources, or material that result in preventing, 
minimizing, or limiting the potential for recurrence of a problem. 

Finding A conclusion based on facts established during the assessment/inspection 

by the investigating authority. 

Lessons Learned Knowledge or understanding gained by experience. The experience may 
be positive, as in a successful test or mission, or negative, as in a mishap 
or failure. A lesson must be significant in that it has real or assumed 
impact on operations; valid in that it is factually and technically correct; 
and applicable in that it identifies a specific design, process, or decision 
that reduces or limits the potential for failures and mishaps, or reinforces a 
positive result. 


Observation A factor, event, or circumstance identified during the 

assessment/inspection that did not contribute to the problem, but if left 
uncorrected has the potential to cause a mishap, injury, or increase the 
severity should a mishap occur. 


Problem 


The subject of the independent technical assessment/inspection. 


Recommendation An action identified by the assessment/inspection Team to correct a root 

cause or deficiency identified during the investigation. The 
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recommendations may be used by the responsible C/P/P/O in the 
preparation of a corrective action plan. 

Root Cause Along a chain of events leading to a mishap or close call, the first causal 

action or failure to act that could have been controlled systemically either 
by policy/practice/procedure or individual adherence to 
policy/practice/procedure. 


11.0 List of Acronyms 


AA Associate Administrator 

CERB Chief Engineer’s Review Board 

ET External Tank 

GUCP Ground Umbilical Carrier Plate 

ICD Interface Control Document 

IFA In-Flight Anomaly 

IFRs Ice Frost Ramps 

IRT Independent Review Team 

JSC Johnson Space Center 

KSC Kennedy Space Center 

LCC Launch Commit Criteria 

LH2 Liquid Hydrogen 

LM Lockheed-Martin 

L02 Liquid Oxygen 

MSFC Marshall Space Flight Center 

MAF Michoud Assembly Facility 

MMC Martin Marietta Corporation 

MMT Mission Management Team 

MR Material Review 

MRB Material Review Board 

NESC NASA Engineering & Safety Center 

NCFI North Carolina Foam Institute 

NSTS National Space Transportation System 

OCE Office of Chief Engineer 

OML Outer Mold Line 

OMRS Operation and Maintenance Requirements and Specifications 

PAL Protuberance Air Load 

PDL Polymer Development Laboratory 

PRA Probabilistic Risk Assessment 

QC Quality Control 
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S&MA 

Safety and Mission Assurance 

SE&I 

Systems Engineering and Integration 

SLA 

Super-Lightweight Ablator 

SLWT 

Super Lightweight Tank 

SOMD 

Space Operations Mission Directorate 

SRB 

Solid Rocket Booster 

SSME 

Space Shuttle Main Engine 

SURFICE 

Surface Ice Tool 

SSP 

Space Shuttle Program 

TIM 

Technical Interchange Meeting 

TPS 

Thermal Protection System 

USA 

United Space Alliance 

USI 

Use As Inspected 

USR 

Use As Repaired 

V&V 

Verification and Validation 

VAB 

Vehicle Assembly Building 
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Appendix A. NESC Request Form 


NASA Engineering and Safety Center 
Request Form 


Submit this ITA/I Request, with associated artifacts attached, to: n fbcxCCSeC'fl inflSfl.gQV . or to 
NRB Executive Secretary, M/S 105. NASA Langley Research Center, Hampton. VA 23681 


Section 1: NESC Revie m p Board (NRB) Executive Secretary Record of Receipt 


Received (mm/dd/yyyy h:mm am/pm) 
3/5/2007 12:00 AM 

Status: New 

Reference #: 07-005-E 

Initiator Name: William Gerstenmaier 

E-mail: 

Center: HQ 


will iam . h .gersten maier@ na 



sa.gov 


Phone: (202)-3 58-20 15, Ext 

Mail Stop: 


Short Title: STS- 1 1 7 Hail Repair Assessment 


Description: 

This is a quick turn-around assessment 

Source (e.g. email, phone call, posted on web): e-mail 

Type of Request: .Assessment 

Proposed Need Date: 

Date forwarded to Systems Engineering Office (SEP): (mmddvvw himm am /pm): 

Section 2: Systems Engineering Office Screening 
Section 2.1 Potential ITA/I Identification 

Received by SEP: (mm ddAyyy h:mm am pm): 3/5/2007 12:00 AM 
Potential ITA/I candidate? [xlves I 1 No 

Assigned Initial Evaluator (IE): litis has been approved out-of-board by Tim Wilson on 3/5/2007. Tim 

Wilson will be leading this. There is no Initial Evaluation required. 

Date assigned (mnvdd/yvyy): 3/5/2007 

Due date for ITA I Screening (mnvdd/yvyy): 

Section 2.2 Non -ITA/I Action 

Requires additional NESC action (non-lTA I)? ^Yes DNo " ~ 

If yes: 

Description of action: 

Actionee: 

Is follow-up required? [X]Ves | | No If yes: Due Dale: 

Follow-up status/date: 

If no: 

NESC Director Concurrence (signature): 

Section T Initial Evaluation 

Received by IE: (mm/dd/yyyy h:mm am/pm): 

Screening complete date: 

Valid ITA/I candidate? QYes Q No 
Initial Evaluation Report #: NESC-PN- 

it NRB Review Date: I 
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Section 4: NRB Review and Disposition ofNCE Response Report 

ITA/1 Approved: 1 lYes 1 1 No Date Approved: | Priority: - Select - 

ITA/I Lead: . Phone ( ) - , x 

Section 5: ITA/I Lead Planning Conduct, and Reporting 

Plan Development Start Date: 

ITA/I Plan # NKSC-PL- 

Plan Approval Date: 

ITA/I Start Dale | Planned: | Actual; 

ITA/I Completed Date: 

ITA/I Final Report #: NKSC-PN- 

ITA/I Briefing Package #: NESC-PN- 

Follow-Up Required?! lYes I I No 

Section 6: Follow-up 

Date Findings Briefed to Customer: 

Follow-up Accepted: | |Ves P) No 

Follow-up Completed Date: 

Follow-up Report #: NESC-RP- 

Section 7: Disposition and Notification 

Notification type: - Select - | Details: 

Date of Notification: 

Final Disposition: - Select - 

Rationale for Disposition: 

Close Out Review Date: 
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Form Approval and Document Revision History 


Approved: 


NESC Director 

Date 


Version 

Description of Revision 

OHice of Primary 
Responsibility 

Effective 

Date 

1.0 

Initial Release 

Principal Engineers 
Office 

29 Jan 04 
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Appendix B. Performance History of PDL Repairs and Crushed Foam 


Table B-l Performance History of PDL Repairs 


Source 

Description 

Conditions 

Results 

Assessment 

At Di voting Depth Limit of 
Largest Void 

809-9621 PDL 1034 
Application/Processing 
Assessment V&V 

Post-Columbia 
assessment of pre- 
Columbia V&V 

101 horizontal 
pours and 49 
vertical pours 

Internal void formation was deemed to be 
low probability in horizontal applications, 
but higher in vertical and overhead pour 
positions. The actual void size was not 
recorded but was below the 0.5 x 0.5” pre- 
Columbia threshold. 

Points to 0.5” cylindrical 
voids. 

Critical depth for 0.5” cylinder 
is 0.8” deep and results in 
0.005 Ibm divot assuming that 
the entire divot is PDL. 

809-9972 Delta 
Validation of Longeron 
Plug Pull Restoration 
V&V 

New Validation 
Data with 
Dissections 

50° from horizontal 
and 5 accepted 
pours. 

3 of the 5 accepted pours had voids 

Range of slot sizes. 
Maximum found 0.4”. 

Critical depth for 0.4” slot is 
0.8" and results in a 0.004 
Ibm divot from 0.1” slot, 0.005 
Ibm divot from 0.2” slot, and 
0.007 Ibm divot from 0.3” slot 
assuming entire divot is PDL. 

809-9972 TPS 
Process Assessment 
Summary, Maximum 
Expected Defect 
Determination for Fly- 
As-ls ET TPS 
Hardware Dissection 

Partial dissection 
of TPS on 4 
external tanks 

19 repairs 
dissected 

15 process cylindrical defects and 25 
process slot defects. Largest process 
cylinder was 0.7”. Largest process slot was 
0.4”. 

Largest process cylinder 
was 0.7”. Largest process 
slot was 0.4” 

Critical depth for 0.7” cylinder 
is 0.9” deep and results in 
0.008 Ibm divot assuming 
entire mass is PDL. 
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Source 

Description 

Conditions 

Results 

Assessment 

At Di voting Depth Limit of 
Largest Void 

809-9440 TPS 
Process Assessment, 
Maximum Expected 
Defect 

Dissection data 
of PDL repairs of 
plug-pulls 



Geometric Cylinders = 1.5” 
Processed Cylinders = .98” 
Geometric Slots = .84” 
Processed Slots = .56” 


826-2048-85, "Effects 
of PDL-1034 Repairs 
on Downstream 
Recession of NCFI 24- 
124", 1997 

NCFI with 
standard PDL 
Repairs 

Hot gas test 

Repaired and downstream areas showed 
acceptable performance 

Verification of standard 
PDL repair 


No Test Report 
Number, " 
Woodpecker Divot 
Testing in the 
Improved Hot Gas 
Facility 

NCFI with 
Standard PDL 
repairs and sand 
and blends. 5 
sand and blends 
tested 

Hot gas test 

Normal recession - no debris generation 

Verification of standard 
PDL repair. Verification of 
sand and blend if damaged 
area is at bottom of sand 
and blend. 


NASATM 110857, 
“Debris/lce/TPS 
Assessment and 
Integrated 

Photographic Analysis 
of Shuttle Mission 
STS-70,” 1995 

Test panels with 
multiple PDL 
repairs of varying 
sizes 

Hot gas test 

No significant debris loss or evidence of 
unacceptable recession was observed 

Verification of PDL repairs 
in areas having multiple 
damage sites 


809-8807, “ET-124 
Crushed Foam 
Testing”, Addendum 6, 
no date 

Requalification of 
PDL material for 
ET-124 due to 
new supplier. 
Test panels with 
multiple PDL and 
sand and blend 
repair areas. 

Hot gas test 

PDL requalified. Demonstrated acceptable 
performance of multiple PDL and sand and 
blend repairs in the L02 ascent 
environment. 

Verification of PDL and 
sand and blend repairs in 
areas having multiple 
damage sites 
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Table B-2 Performance History of Crushed Foam 


Source 

Description 

Conditions 

Results 

Assessment 

ETTR-62 1 , “Hail 
Damage Simulation on 
NCFI24-124” 

27 2.4” net NCFI 
panels 

Steel balls used to 
simulate hail 
Various angles of 
incidence 
Also machined 
panels 

ambient 

1” steel balls created 0.6” deep crushed foam at 
bottom of cavity. Smaller than maximum STS-117 
hail size. 

Shows that crushed foam exists 
below visual zone 

809-9910, Thermal 
Vacuum Testing of 
Crushed Foam with 
Cryogenic Backface,” 
Apr, 2006 

Crushed foam via 
walking loads 
through the walking 
mats 

vacuum/ IR 

No debris loss from damaged areas. 300# load 
applied over large area (~4”0). 

Limited - invisible damage zones 
on ET-124 are caused by a very 
different process 

809-9954, “Thermal 
Evaluation of Crushed 
ET TPS in the 
Improved Hot Gas 
Facility,” Apr 2006. 

6 net spray and 
machined iNCFI 
panels, each with 6 
crush regions - 3 
1x1/2” ellipses and 3 
4” circles, crush 
depth 5, 15, and 
20% 

Also BX, PDL 

Hot gas 

Single foam loss from 20% crush area on machined 
panel. Crush mechanism may not be same as for 
hail 

Single foam loss event from 
crushed foam 
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Source 

Description 

Conditions 

Results 

Assessment 

809-9890, “Red Dye 
and Shearography 
Evaluation as 
Determination by 
Tensile Strength of 
Crushed Foam, " May 
2006. 

Net spray NCFI 
panels subjected to 
crush loads from 4” 
diameter disk. 

Also Machined 
panels, BX, PDL 

ambient 

NCFI is sensitive to crushing 

Limited - this was a detection test 
but it did show that NCFI is 
vulnerable to crushing 

809-8603, “Thermal 
Vacuum Testing: 
Recession 
Characterization of 
Crushed TTPS,” May 
2006. 

6 net spray NCFI 
panels, each with 6 
crush regions - 3 
1x1/2” ellipses and 3 
4” circles with 
radiused edges, 
slow crush depth 5, 
15, and 20%. Also 
Machined panels, 
BX, PDL 

Vacuum/IR 

Normal recession - no debris generation. Crush 
mechanism not be same as for hail - speed 

Some relevance to undetected 
damage 

809-9954, “IFA, AC-14, 
Quick Look, Evaluation 
of Working-Walking 
Loads 

1 .2” thick net spray 
NCFI panels 
subjected to slow 
crush loads from 3” 
diameter disk and 4” 
sphere 

vacuum/IR 

Foam losses from panels with disk loads of 150 
and 200 lbs. Load applied slowly over large area 

Two di voting events from crushed 
foam 

809-9655, “Ice/frost 

Characterization 

testing” 

5 net spray NCFI 
panels 1” and 2” 
deep crushes (25% 
and 15%, 
respectively) 


Ice formed at crushed foam 

Ice/frost can result from damaged 
foam 


Source: Wilson, T., "NESC STS-1 17 Hail Damage Repair Assessment, Project Status," Mar 21, 2007. 
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Appendix C. List of Documents, Tests and Analyses Reviewed by the NESC 


Document Type 

Document Description 

Document 

Impacted 

Date Last 
Updated 

Overview 

Presentation 

Presentation summarizing the 
damage, repair approach and 
verification 

Overview Briefing 

Info Only 

4/27/07 1 

Applicable Documents 

Level II 
Requirement 

Flight and Ground System 
Specification 

NSTS 07700 Volume X - Book 1 

Info Only 

4/27/07 

Level II 
Requirement 

Ice/Debris Inspection Criteria 

NSTS 08303 

Info Only 

4/27/07 

Level II 
Requirement 

Expected Debris Generation 
and Impact Tolerance 
Requirements, Groundrules, 
and Assumptions 

NSTS 60559 

Info Only 

4/27/07 

ICD 

Moldline & Protuberances ICD 

ICD-2-00001 

Info Only 

4/27/07 

ICD 

Space Shuttle/Launch Pad & 
Platform ICD possible 

ICD-2-0A002 

Info Only 

4/27/07 

Level II 
Requirement 

OMRSD Requirements - 
Operation & Maintenance 
Requirements & Specification 
Document 

NSTS 08171 -File IV -ET 
OMRDSs 

File IIVoll.Vol3.Vov4 

Info Only 

4/28/07 
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Level II 
Requirement 

LCC - Launch Commit Criteria 

NSTS 16007 -Section 17- 
External Tank Subsvstems 

Info Only 

4/28/07 

Environments 

Documentation of Mission 
Specific Heating for Launch 
Probability 

SE&I Thermal Environments 

Info Only 

(available on request) 

5/2/07 

End Item 

Specification 

(EIS) 

ET Contract End Item 
Specification 

CPTO1M09A. END ITEM (CEl) 
SPECIFICATION - PART I 

Info Only 

4/28/07 

Verification Plan 

ET requirement verification 
plan 

TM01 

Info Only 

4/28/07 

Affected Documents 

Level II 
Requirements 

Applicable Environments 
Changes 

Thermal Environments - no 
change 

No 

5/2/07 1 

End Item 

Specification 

(EIS) 

Revisions to Contract End Item 
Specification 

N/A 

No 

5/2/07 

Verification Plan 

ET requirement verification 
plan 

ET-124 Repair affected 
Verification Matrix 
L02 Tank Acreage TPS 

• T521C-ET124 
Intertank Acreage TPS 

• T522C-ET124 

Impacted 

5/1/07 

FMEA/CIL 

Changes to baseline / violations 
as the result of ET-124 repair 

CIL Item Monitome Discussion 

Impacted 

4/27/07 

ICD 

Changes to baseline / violations 
as the result of ET-124 repair 

Waiver IRN if required 

TBD 

ECD ? 

Impact will be 
pending final 
assessment of 
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As-Built 

OMRSD 

Changes to baseline / violations 
as the result of ET-124 repair 

N/A 

No 

4/27/07 

LCC 

Changes to baseline / violations 
as the result of ET-124 repair 

N/A 

No 

4/27/07 

Hazards 

Changes to baseline / violations 
as the result of ET-124 repair 

• T.02 - Loss of ET Thermal 
Protection Svstem (changes) 

• T.04 - ET Ice Debris/Damage 
(changes) 

Impacted 

5/02/07 

Planning update - 
not available for 
RVR 

Available for FRR 

MUAs 

393D 

Undated MUA for PDL 1034 
fNCFI 26-0071 

Info Only 

4/27/07 

MUAs 

042 1C 

Undated MUA for BX-265 

Info Only 

4/27/07 

Required Testing 

Test Plan /Report 

Changes to TPS materials - 
PDL-1034 (NCFI 26-007) 

Plan 

• MS-06-040 

• MS-06-040 Rev 1 

• MS-06-040 Rev 1. Add 1 
Report 

• 809-8544-1 

• 809-8544-2 

Impacted 

4/28/07 

Test Plan / Report 

Changes to TPS materials - 
BX-265 (Polyol source change) 
Qualification test plan/report 

Plan 

• 809-8600 Rev A 

• 809-8600 Rev A Add 1 

• 809-8600 Rev A Add 2 
Report 

• 809-8601 R1 

Impacted 

4/28/07 
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Forward Ogive Repair Testing 

Test Plan /Report 

Demonstration 

Plan 

• 809-8802 
Report 

• 809-8803 

• Flash Report 

Impacted 

5/2/07 

Test Plan /Report 

Hot Gas Testing 

Plan 

• 809-8804 

• 809-8804 Amendment 1 

• 809-8804 Amendment 2 
Report 

• 809-8805 

• Flash Report 

Impacted 

4/28/07 

Repair Testing 809-8807 
Combined Flash Report - All Addenda 

Test Plan /Report 

Addendum 1, Sand and Blend 
Test 

• 809-8807. Add. 1 (Plan) 

• 809-8808 -ET 124 HAIL 
DAMAGE CRUSHED 
FOAM TESTING - 
Addendum 1 Sand & Blend 
Thermal Testing (Report) 

Impacted 

4/28/07 
Report ECD ? 

Test Plan /Report 

Addendum 2, Hail Damage 
Simulation 

• 809-8807. Add. 2 (Plant 

• 809-8836 -ET 124 HAIL 
DAMAGE CRUSHED 
FOAM TESTING - 
Addendum 2 Hail Damage 

Impacted 

4/28/07 
Report ECD ? 
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Simulation (Report) 



Test Plan /Report 

Addendum 3, Intertank Hot Gas 
Recession Test 

• 809-8807. Add. 3 tPlanl 

• 809-8837 -ET 124 HAIL 
DAMAGE CRUSHED 
FOAM TESTING - 
Addendum 3 Intertank Hail 
Damage Hot Gas Test 
(Report) 

Impacted 

4/28/07 
Report ECD ? 

Test Plan /Report 

Addendum 4, Dynamic vs 
Static Foam Crushing Test 

• 809-8807. Add. 4 tPlanl 

• 809-8838 -ET 124 HAIL 
DAMAGE CRUSHED 
FOAM TESTING - 
Addendum 4 Dynamic vs. 
Static Crushed Foam Hot Gas 
Test (Report) 

Impacted 

4/28/07 
Report ECD ? 

Test Plan /Report 

Addendum 5, Barely Visible 
Damage “BVD” Hot Gas Test 

• 809-8807. Add. 5 IPlanl 

• 809-8839 -ET 124 HAIL 
DAMAGE CRUSHED 
FOAM TESTING - 
Addendum 5 Barely Visible 
Damage Hot Gas Test 
(Report) 

Impacted 

4/28/07 
Report ECD ? 

Test Plan /Report 

Addendum 6, PDL Repair Hot 
Gas Test 

• 809-8807. Add. 6 fPlanl 

• 809-8807. Add. 6 A1 fPlanl 

• 809-9940 -ET 124 HAIL 
DAMAGE CRUSHED 
FOAM TESTING - 
Addendum 6 PDL Repair Hot 

Impacted 

4/28/07 
Report ECD 
5/6 
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Gas Test (Report) 



Test Plan /Report 

Addendum 7 & 8, Barely 
Visible Damage “BVD” Icing 
and PDL Thermal Vac Test 

• 809-8807. Add. 7/8 IPlanl 

• 809-8841 -ET 124 HAIL 
DAMAGE CRUSHED 
FOAM TESTING - 
Addendum 7/8 Barely Visible 
Damage / PDL Repair 
Thermal Vac Test (Report) 

Impacted 

4/28/07 
Report ECD 
5/11 

Test Plan /Report 

Addendum 9, Intertank Thermal 
Test 

• 809-8807. Add. 9 IPlanl 

• 809-8842 -ET 124 HAIL 
DAMAGE CRUSHED 
FOAM TESTING - 
Addendum 9 MSFC 
Simulated Hail Damage 
Thermal Vac Test (Report) 

Impacted 

4/28/07 
Report ECD ? 

Test Plan /Report 

Addendum 10, Hot Gas Test 

• 809-8807. Add. 10 IPlanl 

• 809-8843 -ET 124 HAIL 
DAMAGE CRUSHED 
FOAM TESTING - 
Addendum 10 L02 Tank Hail 
Damage Hot Gas Test 
(Report) 

Impacted 

4/28/07 
Report ECD ? 

Plug Pull Failure Testing - Cat 3 

Test Plan /Report 

Test #2: Plug pull re-core 

LWR #21958 
Flash Report 

Impacted 

05/02/07 
Report ECD 5/7 

Test Plan /Report 

Test #3: Insecticide 

LWR #21953 
Flash Report 

Impacted 

05/02/07 
Report ECD 5/7 
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Related Analyses 

Analysis Results 

Thermal Analysis - 
Documentation of analysis 
results that support the as-built 
configuration 

Launch Probability 

• 4140/T-07-3016 - Assessment 
of Launch Probability with 
ET-124 Hail Damage 

Icing Test Results 

• 4140/T-07-3017 - Sand/Blend 
Criteria for Icing for ET-124 
Repair Assessments for the 
L02 Tank 

Heat Leaks 

• 4140/T-07-3013 - 
Documentation of L02 Heat 
Leak Assessment for ET-124 
Hail Damage 

L02 acreage 

• 4140/T-07-3014 - Reauest for 
Stress Assessment of L02 
Tank Temperatures for ET- 
124 Hail Damage 

Pencil Sharpener Repair 

• 4140/T-07-3008 - Thermal 
Analysis of Pencil Sharpener 
Repair at Station 372 on ET- 
124 

Pencil Sharpener Repair 

• 4140/T-07-3018- ET-124 

Impacted 

05/03/07 
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Hail Damage L02 Ogive 
Structural Temperature at XT 
387 with BX/NCFI Repair 

Pencil Sharpener Repair 

• 4140/T-07-3021 - ET-124 
Hail Damage L02 Ogive 
Structural Gradients for Divot 
Analysis at XT 375 with 
BX/NCFI Repair 

Pencil Sharpener Repair 

• 4140/T-07-3019 - 
Aerothermal Testing of the 
BX-265 over NCFI 24-124 
Forward Ogive Machined 
Area (Pencil Sharpener Area) 
Repair Due to Hail Damage 
on ET-124 

Intertank-structure 

• 4140/T-07-3012 - Thermal 
Analysis of Intertank Zones 
for ET-124 Damage 
Assessment 

L02 entry analysis 

• 4140/T-07-3020 - ET Reentry 
Analysis for ET-124 L02 
Tank Hail Damage 

Methodology for Cavities 

• 4140/T-07-3010 - Sand/Blend 
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Criteria for Heating for ET- 
124 Thermal Analysis 
Assessments 



Analysis Results 

Stress Analysis - 
Documentation of analysis 
results that support the as-built 
configuration 

4130-07-01 - ET 124 Hail Damage 
L02 Tank Structural Assessment 
4130-07-02 - ET 124 Hail Damage 
Intertank Structural Assessment 
4130-07-03 - ET 124 Hail Damage 
Composite Assessments 

Impacted 

ECD 

Analysis Results 

Propulsion Analysis - 
Documentation of analysis 
results that support the as-built 
design 

4160/P-07-4002 - Assessment of 
ET-124 Minimum Foam Thickness 
Requirements on L02 Tank Ullage 
Pressure and EOR Propellant Load 

Impacted 

05/03/07 

Other Information 

Historical Documents 

Review Data 

ET-124 Hail Damage TIM 

#1 ET-124 Hail Damage Technical 
Interchange Meeting 3/14/07 

Info Only 

5/02/07 

Presentations - 
TSC/CERB 

ET-124 Hail Damage TSC / 
CERB briefings 

TSC Briefings 

4/5/07 ET-124 Component 
Dispositions 

4/5/07 Spray Demonstration results 

CERB Briefings 

3/6/07 ET Hail Damaee 
assessment 

3/12/07 ET-124 Reoair 
Assessment Approach 

3/12/07 Red Dve and crushed foam 

Info Only 

4/28/07 
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3/27/07 ET-124 PPL Repair 
Reduction 

3/27/07 ET-124 Hail Damage 
Sampling Results 

3/27/07 Planned Testing to 
Support Disposition of ET-124 
Hail Damage 

3/27/07 ET-124 L02 Tank 
Temperature Assessment 

3/27/07 LQ2 Tank TPS Thickness 
requirements delta 

3/27/07 ET-124 TPS Repair 
Impacts to STS-1 17 LQ2 Tank 
EOR Propellant Load - PSIG 

3/27/07 Thermal Sand and Blend 
criteria for icing 

3/27/07 ET-124 LQ2 Tank 
Thickness Map Mission specific 

3/27/07 ET-124 New BX Spray 

3/27/07 ET-124 Intertank 
Assessment Plan 

4/12/07 809-8804 Hot Gas Brief 
Report 

4/12/07 ET-124 Sand and Blend 
Criteria for Icing 

4/18/07 ET Pencil Sharpener 


NESC Request No.: NESC 07-005-E 


0 

NASA Engineering and Safety Center 
Technical Report 

Document #: 

RP-07-47 

Version: 

1.0 

Title: 

STS-117 Hail Damage Repair Assessment 

Page #: 

73 of 113 




Repair demo testing update 

4/18/07 Hail Damage Testing 
update 

4/18/07 ET Proiect proposed ice 
ball testing 



Presentations - 
TSC/CERB 

NCFI-007 (PDL-1034 
replacement) TSC / CERB 
briefings 

01/1 1/07 TSC: Review NCFI -007 
(PDL-1034) material qualification 
test data 

03/06/07 TSC: Review NCFI-007 
(PDL-007) material qualification 
test data 

04/05/07 TSC: Review NCFI-007 
(PDL-1034) cryoflex monostram 
test data 

Info Only 

4/30/07 

Presentations - 
TSC/CERB 

BX-265 re-qualification TSC / 
CERB briefings 

1 1/30/06 TSC: Review chemical 
and mechanical property data of 
BX-265 material with Polvol 
source change 

01/08/07 TSC: Review BX-265 
material with Polvol source change 
hot gas test data 

02/16/07 TSC: Review BX-265 
Lot 1 Thermal Vac Thermal 
Conductivity data and Lot 
2Mechanical and Hot Gas 

03/20/07 TSC: Review BX Lot 2 
Thermal Vac and Thermal 

Info Only 

4/30/07 
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Conductivity data 



MPP 

AFT OGIVE HAIL DAMAGE 
BX-265 REPAIR @ KSC (ET- 
124) 

(Montana demonstration) 

ARK07014M10 Rev 2 

Info Only 

4/28/07 

MPP 

FWD OGIVE HAIL DAMAGE 
BX-265 REPAIR @ KSC (ET- 
124) 

ARK07014K1 Rev 2 

Info Only 

4/28/07 

Reference Info 

ET-124 Hail Damage Repair 
Data 

ET-124 Repair Data bv Tvne 

Info Only 

5/02/07 

Repair Summary 

ET-124 Hail damage PDL 
repair calculations 

Defect Data Summary 

Info Only 

4/30/07 

Reference Info 

Environment Update Process 

PRCBD CR S071024LW 
Induced Environments Transmittal 
Process 

Info Only 

4/30/07 

Reference Info 

ET Project - Design Values for 
Non-Metallic Materials 

809-9600 

Info Only 

4/30/07 

Reference Info 

Foam Losses 

L02 Acreage Foam Loss Summary 

Info Only 

5/01/07 

Reference Info 

Composite Intertank Door 
Ultrasonic Inspection Results 

Report 2007-CB-01 

Info Only 

5/02/07 

Reference Info 

Composite GH2 Pressline 
Fairing Ultrasonic Inspection 
Results 

Report 2007-CB-02 

Info Only 

5/02/07 

Reference Info 

Thermography of the 
Composite Nose Cone on ET- 
124 

Report 2007-CB-03 

Info Only 

5/02/07 

Presentation - 
NASA Senior 
Management 

ET-124 Hail Damage 
Assessment and Verification 
Approach 

ET-124 Hail Damage Assessment 
and Verification Approach 
Presentation 

Info Only 

5/03/07 
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Review 





Actions for Reference 

ET PRCBD Actions 

PRCBD S063595B (1-1) - ET 
Foam Minimum Thickness 
Requirements (STS-1 17) - 
Perform a LOX Tank heat leak 
assessment based on the final 
repair configuration of ET-124. 
Report to the SICB. 

PRCBD S063595B and Response 
(Planned Presentation to SICB on 
5/8) 

Info Only 

4/30/07 
Response ECD 
5/8 

SEPARs 

Transmittal of request to 
perform repair (from LM / KSC 
to USA /KSC) 

All SEPARS with attachments 

Info Only 

(available on request) 

4/30/07 

Action Requests 

Transmittal of defect 
disposition and rationale for 
acceptance of disposition (from 
LM / MAF to LM / KSC) 

All ARs 

Info Only 

(available on request) 

4/30/07 
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Appendix D. (Reserved) 
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Appendix E. Process Demonstration for Multiple PDL Pours per 

Single Syringe Load 

On March 26 and March 27, 2007, USA performed two “process demonstrations” consisting of 
PDL pours on BX-265 foam test panels mounted vertically in a different area of the VAB than 
the actual ET work area. The primary goal of the demonstration was to develop and validate a 
revised method of PDL repairs using multiple PDL pours from a single syringe loaded with PDL 
foam. The intent of the new method was to increase overall efficiency of the repair process 
without sacrificing quality. The efficiency increase was desirable in order to get the repairs to a 
“production mode” that would support the milestones in the repair schedule. The results of the 
process demonstration were summarized in a United Space Alliance (USA) white paper provided 
to NESC on April 16,2007. 

TPSB SS20-613 and TPSB SS20-614 were the procedures developed to support the PDL pours 
in two test panels with 17 holes each. The process demonstration verified the methods for the 
production mode of PDL repairs at KSC. 

The stated purpose of TPSB SS20-613 was to “verify PDL cream time in the VAB environment 
and determine syringe size and multiple holes proficiency.” (reference: USA white paper, April 
16, 2007 ). The results of the TPSB SS20-613 procedure included the following: 

- Determined a 1.7 cc of PDL foam in the syringe per cubic inch of estimated repair 
volume 

- Determined that 3 holes in close proximity was the limit for one syringe 
Selected the 10 cc syringe size to support a three-hole injection 

- Verified the PDL cream time supported the syringe method and a three-hole injection 
series 

During TPSB SS20-613, mold failures were experienced from foam lifting the mold off the ET 
surface, which raised concerns regarding sub-surface voids. As a result, only hard molds were 
allowed in the repair methods. A double ring of sealant tape (“dum dum” putty) was also used to 
secure the hard mold to the ET. The sealant tape was a contributor to rework activities (i.e. sand 
and blend) when removal of the tape also lifted some of the surface ET foam (the “rind”) during 
mold removals. 

The stated purpose of TPSB SS20 614 was to “demonstrate the multiple holes pour and to verify 
injection parameters and that lessons learned (from TPSB SS20 613) were incorporated” 
(reference: USA white paper, April 16, 2007)) into the final PDL repair procedure. Specific 
objectives of this demonstration included verifying that the subsurface void criteria were not 
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violated and verifying technician proficiency (in addition to the proficiency already 
demonstrated during the proficiency already demonstrated during the certification/recertification 
process). The results of the TPSB SS20-614 procedure included the following: 

17 of 17 test PDL repairs were free of subsurface voids. One of 17 repairs was an under 
fill. 

- Verified by dissection that no subsurface void criteria (0.5 inch maximum) was violated 
on all test samples. 

“Buy copies” of TPSB SS20-613 and TPSB SS20-614 were provided to NESC. No dissection 
measurements or photographs were collected as part of TPSB SS20-613 or TPSB SS20-614. 
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Appendix F. NESC Physically Based Non-Dimensionalization of 

Icing Probability 

Rationale 

The External Tank Project devised a method of sorting the hail damage locations into deeper- 
than-normal sand and blend (S&B) and PDL repairs with a goal of minimizing the number of 
PDL repairs on the tank. This sorting was performed by identifying the icing potential of each 
prospective S&B repair. Those that had an icing probability of greater than 1 5 percent for a May 
through October launch were selected by the project to be repaired using PDL. To enable this 
sorting, the project set up an icing test in a natural convection chamber. A box fan was included 
in the test to allow for some forced convection. Liquid nitrogen-backed foamed panels with 
cavities of different depths and shapes were tested to identify the icing limits. 

The results of the testing are shown in Figure J-l along with the 327 damage locations included 
in the sorting. The line labeled as “test derived with wind” was developed directly from the 
tipping points identified in the icing tests. It was used to sort the repair sites into PDL and S&B, 
resulting in 26 suggested PDL repairs and 301 S&B repairs. Also, 3 repairs that fell into the 
S&B region were re-designated as PDL by the ET Project owing to a feeling of increased icing 
risk at these locations and depths. 

A NESC preliminary investigation of the historic at KSC weather indicated that the 15 percent 
icing probability (85 percent launch probability) cutoff most likely coincided with wind speeds 
that would cause the heat transfer on the LO 2 ogive to be dominated by forced convection. This 
raised an issue with directly using the results of a natural convection test with limited forced 
convection to assess icing in the forced convection dominated launch conditions. In addition, it 
was noted that the sorting criteria did not take into account the fact that the new minimum foam 
thickness is 0.8 inch in the no-ice zone vs. 0.6 inch in the ice limitation zone. Therefore, the 
icing potential of the prospective S&B repairs on all zones of the tank cannot be represented 
using defect depth and foam thickness as the sole variables. Because of these factors, the NESC 
performed a physics-based non-dimensionalization of the icing potential to allow the relative 
icing potential of all prospective S&B repairs to be assessed. 
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PDL Repairs 

I Baseline estimate from preliminary test data assessment = 26 
Test-derived with wind: 25 + 3 additional in no-ice zone = 28 
Test-derived without wind: 110 


Recommended repairs (3) to 
improve margin in no-ice zone 

• d2D OO 


mCXQCD <$> O O 



m — — • 9m O •0 # <3332> • 


ET-124 Hail Damage Defects 

O No-ice zone = 43 
i Ice-limitation zone = 284 
Total in no-ice & ice-limitation zones = 327 


1.5 1.7 1.9 

Adjacent Foam Thickness - in. 


2.5 


Figure FI - Sand/Blend Criteria for Ice Zones (from Sand and Blend Criteria for ET-124 
Icing Zones, Michelle Guillot. April 12, 2007 


Physics-Based Non-Dimensionalization 

For foam on the oxygen tank at the minimum thickness, tmin, for a given ambient temperature, 
To, , there is a value of the nominal convective coefficient, h nom inai, below which unacceptable 
icing can occur. This defines a critical surface temperature, T sur f a ce, critical- The physical case is 
shown in Figure J-l . 
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Figure F-2 - Physical Layout of Foam Heat Transfer 

The heat transfer through the foam can be characterized by the thermal resistances shown in 
Figure J-2. Here, keff is the effective thermal conductivity of the foam and A is a characteristic 
surface area perpendicular to the heat transfer path. 



Figure F-3 -Thermal Resistances at the Minimum Foam Thickness 
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To maintain the same icing probability as a large flat area at the minimum thickness, the surface 
temperature at the bottom of a sand and blend must be greater than or equal to critical surface 
temperature. A one-dimensional analysis allows the thermal resistances to the bottom of the 
cavity to be represented as shown in Figure J-3, where t is the foam thickness and h is the 
convection coefficient in the bottom of the cavity. 


R, = 


k 9t fA 


T n = -297° F 

°2 


v eff 


Ro = 


t 


hA 


surface, critical 


Figure F-4 - One-Dimensional Thermal Resistances at the Bottom of a Cavity 

To maintain the same critical surface temperature for different foam thicknesses, the ratio of the 
two thermal resistances, R1/R2 must be the same in the two cases. This yields the following 
relationship for the convective coefficient at the bottom of the cavity required for equivalent 
surface temperature 

h - h * min m 

This result is independent of ambient temperature, the critical surface temperature, and the 
nominal heat transfer coefficient. If the convective coefficient at the bottom of the cavity is 
higher than the value calculated by eqn. (1), the icing potential is less than for a flat surface at the 
minimum foam thickness. Conversely, if the coefficient is lower than this value, there is a 
higher icing potential than for a flat surface at the minimum foam thickness. 

At wind speeds exceeding 2 knots, the Reynolds number at the ogive exceeds 5x1 0 5 and the heat 
transfer mechanism on the tank surface is turbulent forced convection. In this case we might 
expect that the convective coefficient at the bottom of a cavity would be characterized by 
■ the local convection coefficient over a nearby flat surface 
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■ the cavity depth 

■ the cavity width 

■ the cavity shape. 

Buckingham’s Pi theorem suggests that we could express the relation in dimensionless form as 


nominal 


= fn 


X 

d’ 


shapej 


1 


y 


( 2 ) 


where X is the cavity width and d is the cavity depth. For similar cavity shapes, the expression 
becomes 


'nominal 


= fn 


LdU 


( 3 ) 


Buckingham’s Pi theorem says that h/h nom jn a i can be expressed as a single valued function of X/d, 
but does not tell us anything about the shape of the curve. However, the physics of the airflow in 
the cavity suggest that for large values of X/d, the cavity approximates a flat surface and h/h nom j na i 
approaches unity. As X/d decreases, the cavity becomes relatively steeper, reducing the airflow 
in the cavity, and reducing h/h n0 minai- 


The ET Project Test Results 


The ET Project test-based recommendations are plotted using the appropriate dimensionless 
groups in Figure 5. The y-axis is the repair-specific convective coefficient ratio calculated from 
eqn. 1 that must be maintained to yield the same surface temperature as for a flat surface at the 
same thickness. Several conclusions are apparent from the figure. First, the ET recommended 
PDL repairs are clustered where the sand and blend repair would be deep and steep (low 1/d) 
and would need to maintain a high fraction of the flat surface heat transfer. Second, the three 
additional recommended PDL repairs 3 4 are in a region where many other locations are also 
recommended for PDL repair. Third, there is some overlap between the recommendations for 
S&B and PDL repair. 


3 Labeled “ET PDL” in the Figure 5. 

4 Labeled “ET Extended PDL” in Figure 5 
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XI d 

Figure F-5 - ET Project Test-Based Repair Recommendation 

To address the repair method overlap, a cutoff line was drawn encompassing all the ET Project- 
recommended PDL repairs as shown in Figure J-6. The line has the expected upward slope as 
discussed in the previous section. 
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Figure F-6 - Repair Sorting and Recommended Augmentation 


Figure J-6 also shows 1 1 repairs that were recommended as S&B by the ET Project that are 
above the dividing line and thus have icing potential that exceeds that of sites that had been 
designated for PDL repair. The NESC recommended to the project that these sites also be 
repaired with PDL. The ET Project accepted the recommendation. 
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Appendix G. NESC Probability of Launch Based on Icing 


Rationale 

The NESC was concerned that, because tests that were not representative of the launch pad heat 
transfer physics were used to establish the PDL vs. S&B repair cutoff , the launch probability 
due to icing could be substantially less than the 85percent May to October requirement. To 
address this concern, the NESC performed a sensitivity analysis of the heat transfer in the sand 
and blend repairs using a 30 year KSC weather dataset. 

Method 


The first step in performing the sensitivity analysis was to obtain the LO 2 ogive average heat 
transfer coefficient as a function of wind speed using HPSim 5 , the ET Project certified heat 
transfer coefficient calculation tool. Using 75°F 6 as the ambient temperature, the ogive heat 
transfer coefficient was calculated for a range of wind speeds. The result is shown in Figure 1 



Figure G-l - Ogive Heat Transfer Coefficient, 75°F Ambient Temperature 


5 HPSim Rev F, Lockheed Martin Michoud Space Systems 

6 75°F is an approximate average for the ambient temperature at KSC between May and October. 
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The flat region at low wind speeds in Figure 1 is the natural convection region. At wind speeds 
above 1 .5 knots, forced convection heat transfer dominates. Because the heat transfer coefficient 
in the dominant forced convection regime is not a strong function of ambient temperature, the 
HPSim output was curve fit to provide a single valued function of ogive convection coefficient 
as a function of wind speed. 

The thermal resistance network for heat transfer on the ogive with a convective boundary 
condition is shown in Figure K-2. Here A is a representative area, t is the foam thickness, h is the 
convective coefficient, keff is the effective foam thermal conductivity, T sur face is the foam outer 
surface temperature, and T x is the ambient temperature. 



Ro = 


t 


Tq 2 = -297° F 


hA 


1 surface 


eff 


Figure G-2 - Thermal Resistance Network for Ogive Heat Transfer 

For a given foam thickness, if the surface temperature is 32°F (the icing limit), the heat flux, q”, 
is 


q' = ]^fL(32 + 297>F (1) 

The corresponding convective heat transfer is 

q'=h(T„-32>F (2) 
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If the convective heat transfer is higher than the value in eqn. (2), the higher heat transfer will 
drive the surface temperature above the freezing point. Therefore, we can use the quantity F as 
an icing indicator where 

F = h(T„-32)>F (3) 

A 30 year KSC weather data base was obtained from MAF. This database includes hourly 
observations of ambient temperature and wind speed. For each observation in the database, the 
convective coefficient curve fit was used to calculate a value of F. The results were then sorted 
on F. This allowed us to find the value of F that coincides with 15percent icing probability 
(85percent launch probability) and to assess the probability distribution of F. 

The sorted wind speed measurements are shown in Figure 3. Three lines are plotted for 3 
separate data subsets owing to Excel limitations. The wind speed probability shows the 1 .5 knot 
wind speed limit for natural convection is exceeded more than 90percent of the time. 



Figure G-3 - May to October Wind speed at KSC 

The probability distribution of the icing parameter, F is shown in Figure K-4. Here, also, the 
data is broken into three data subsets owing to Excel limitations. A single black line is faired 
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through the three data subsets. The value of F that coincides with an 85percent icing launch 
probability is 30 BTU/hr ft 2 . 



percentile 


Figure G-4 - Icing Parameter, F - May to October at KSC 

The choice between sand and blend repairs and PDL repairs for different hail damage locations 
was made based on a presumed relationship between the dimensions of the sand and blend cavity 
and the heat transfer at the bottom of the cavity as was detailed in Appendix F. Figure K-5 
shows the selected relationship as a diagonal black line. If this line reflects the relationship 
between the fraction of flat surface convective coefficient and the geometry parameter, we have 
maintained 85percent launch probability May-October (per the ET analysis). 
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0 1 2 3 4 5 6 8 9 10 11 12 13 14 
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Figure G-5 - Relationship Used to Discriminate between Sand and Blend and PDL Repairs 


Figure 4 gives us a method of assessing the launch probability sensitivity to the location of the 
dividing line in Figure 5. If dividing line is off by a factor of 2, the critical value of F on a flat 
surface would need to double to prevent icing above the 15percent probability level. Figure 4 
shows that if we select the critical value of F as 60 (2x the nominal 15percent level), the icing 
probability is less than 45percent (and the launch probability still exceeds 55percent). Therefore, 
we conclude that the selected sand and blend criteria yields a relatively robust icing launch 
probability. 
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Appendix H. NESC Cryogenic Moisture Uptake Core Study 


NESC Icc Mitigation Assessment 
Cryogenic Moisture Uptake (CMU) Core Study 

Introduction: CMU Core Study 

Material weight gain due to water vapor condensation and permeation under cryogenic 
conditions was a major consideration of candidate insulation materials. It was deemed critical to 
understand where the water/ice resides within the foam. If a weight gain occurs in the edd 
section of foam then ice/frost would be present and liquid water would be present in the warmer 
sections A test method was developed dial would ulili/.e the CMU test met hodology followed 
by dissection and monitored weight loss to determine water location within the foam. This 
involved taking multiple core specimens from a foam sample soon after removal from the CMU 
test. The cores were cut into sections to determine if weigh loss/gain occurred in the warm 
surface, in (lie middle region, or the cold surface (see lest method). 


Test Method: CMU Core Study 


Standard CMU tests were conducted with the exception that the test specimen was not 
removed to monitor weight gain over time. The specimen was only removed on the 8 th hour 
of cold soak to measure the weight followed by dissection and weight measurement of 
dissected core specimens. Figure 1 below shows approximately how the core sections were 
taken. 



8 " 


Main e 1. Pictured on the left is the core tool and cored practice sample. CMU core diasram on the right. 
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WBT = 293K (68°F) 

Figure 2. CMU core direction diagram. Section C contains the cold boundary temperature (CHI’) and 
section A contains the warm boundary temperature (WBT). 


A core specimen was drilled out and carefully dissected using a long razor knife. Each core 
was cut into 3 sections of approximate equal depth. Each section was labeled relative its 
position during CMU testing. For instance, l A would be the warm side of core one, 3B 
would be the middle section of core three, and 4C would be the cryogenic side of core four. 
Each core section was dissected, labeled, and weighed immediately after being cut from the 
main 8" diameter sample. Once all samples were weighed they were then placed into a dry 
nitrogen purged chamber over night. The samples were then re-weighed to determine 
moisture uptake. Sample total dry weight was verified to the initial dry weight before test. 

Results: CMU Core Study 

The majority of weight gain was determined to be in the warm section A portion of all cores 
followed by section B. and finally the least amount of weiglit gain was found in cold section 
C. The net spray surface of NCFI 24-124 foam materials behaved differently from that of 
the machined surface of BX-265. This may have been due the way the samples were cut. 
Section A took the high points of the rind surface into consideration for dissection. This 
implies that sections B and C were expanded toward the warm surface (had greater depth) 
and may therefore explain the decrease in weight of section A relative to the increases in 
sections B and C. The most surprising results were from the mostly open cell OFT flexible 
polyimide foam sample (-30% closed cell). The unrestricted mass transport of water vapor 


2 of 1 8 


NESC Request No.: NESC 07-005-E 


# 

NASA Engineering and Safety Center 
Technical Report 

Document #: 

RP-07-47 

Version: 

1.0 

Title: 

STS-117 Hail Damage Repair Assessment 

Page #: 

93 of 113 


meant the weight increase could happen through out the thickness of the foam. Unrestricted 
mass transport did not change the distribution of water to any great degree; reasons why can 
be found in the discussion section. 
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Table 1. Cryogenic moisture uptake and core data of Baseline I IX -265 loam. The foam was not weathered 
Core section A data or interest is highlighted in yellow. Final dry weight of the “sample" is the sample left 
after coring (sample with holes). The sum of the “sample” dry weight with all the core sections comes close to 
the initial weight; some foam is lost during dissection. 
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Figure 3. Cry ogenic moisture uptake and core data of Baseline BA -265 foam. Foam core sections w ere 
w eighed after 8 hours of CMC then re-weighed after drying over night in a dry nitrogen purged chamber. 
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i 

figure 5. Photograph or 15 mouth weathered BX 265 foam immediately alter all core sections were taken. 
Note that ice from CMU testing was still present. 
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Figure 6. Photograph of dissected cores of BX-265 15 month weathered after drying. 
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Table i. Cryogenic moisture uptake and core data of net spray baseline NCFI 24-124foam. The foam was 
not weathered. Core seclion A data of interest is highlighted in yellow. Final dry wcighl of the “sample” is 
the sample left after coring (sample with holes). The sum of the “sample’ 1 dry weight with all the core 
scclions comes close to the initial w eight; some foam is lost during disscclion. 
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Figure 7. Cry ogenic moisture uptake and core data of baseline net spray NCFI 24-124 foam. Foam core 
sections w ere w eighed after 8 hours of CMC then re-w eighed after drying over night in a dry nitrogen purged 
chamber. 
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Table 4. Cryogenic moisture uptake and core data of 15 month weathered net spray NCFI 24 124 foam. Core 
section A data or interest Ls highlighted in yellow. Final dry weight of the “sample’' Is the sample left after 
coring (sample with holes). The sunt of the “sample” dry w eight w ith all the core sections comes close to the 
initial weight; some loam is lost during dissection. 
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Figure 8. Cryogenic moisture uptake and core data of 15 month weathered net spray NCFI 24-124 foam. 
Foam core sections were weighed after 8 hours or CMC then re-weighed after drying over night in a dry 
nitrogen purged chamber. 
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Figure 9. Photograph of 15 month weathered net spray NCF1 24-124 after coring and drying overnight. 
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Figure 10. Photograph of dissected cores of net spray NCF1 24-124 15 month weathered samples after 
drying. 
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Tahle 5. Cryogenic moisture uptake and core data of 2 year indoor aged GFT flexible polyimide foam. The 
foam was not weathered. Core seclion A data of interest is highlighted in yellow . Final dry weight of the 
“sample" is the sample left after coring (sample w ith holes). The sum or the “sample” dry w eight w ith all the 
core sections comes close to the initial w eight; some foam is lost during dissection. 
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Figure 11. Cryogenic moisture uptake and core data of 2 year indoor aged GFT flexible polyimide foam. 
Foam core sections w ere weighed after 8 hours of CMU then re-weighed after dry ing over night in a dry 
nitrogen purged chamber. 
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Figure 12. A p hotograp Its of GFT flexib le p clyimide foam immediately after removal from Ciyugenic 
Moisture Uptake test apparatus. Note the ire, 'frost and water drops evident on the warm surface. 
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Figure 13. A photograph two minutes after removal from CMC apparatus the first core was drilled out. 
Note the ice frost section. 
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warn) surface middle cryo surface 


Figure 16. Core data of all samples, ('ore data strongly indicates that though samples had different amounts 
of water weight gain how the water w as distributed was remarkably similar. The 5th core for GFT was 
omitted because it was deemed an outlier. 


Discussion: CMU Core Study 

Test results indicate that weathered foams have higher water/ice weight gain. Based on 
these results a larger percentage of the water/ice weight gain remains near the surface 
(section A) for the weathered versus non-weathered foams tested, BX-265 is better in this 
regard, because there is very little weight in sections B and C, though more extensive 
studies are required to make such conclusions. 

Test results indicate that water distribution throughout the thickness of “closed cell” foam 
samples in comparison to “open cell” foam samples was similar. Frost formation and liquid 
water condensation arc simply a function of dew point, surface temperature, and surface 
area. Water frost deposition does change the local humidity level within the foam thus to 
maintain equilibrium humidity conditions water vapor does need to transport to the 
localized area with reduced humidity, litis mass transport seems to be a small driver as 
indicated by the data (Figure 16.). Though the GFT foam has the ability from a mass 
transport point of view to gain vveigltt in cryogenic section C and middle section B that does 
not occur. What probably occurs in these cold sections is that all of section C is below 
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freezing and most of section B is below freezing thus only low density frost can form. 
Testing is underway to verify this hypothesis. One implication of this hypothesis is that 
hard ice formation might not be such a problem within open cell foam materials. Hard ice 
will only form at the melt freeze line and requires a decrease in ambient temperature or a 
decrease in cryogenic temperature. Any decrease in temperature will drive the freeze line 
out toward the warm surface thus freezing any liquid condensate. In fully hydrophobic 
foam systems small spheres of ice would likely form inboard of an outboard advancing 
freeze line. Tests are under development to characterize type of ice formation within foam 
systems. 
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Appendix I. NESC Expected PDL 1034 Mass Loss Assessment 


Summary 

One of the facets of the flight rationale for STS-1 17 is the probability risk assessment (PRA). 
This assessment uses a Monte Carlo technique to calculate the risk to the Orbiter from the 
expected level of ascent debris. Once the 900+ PDL 1034 repairs were completed on the tank, 
the values for the maximum expected ascent foam loss from each repair could be developed. 

The ET Project developed a database of maximum expected foam loss from the PDL 1034 
repairs using the following procedure: 

1 . The mass of each PDL 1034 repair was calculated from the repair height, width, and 
length assuming a brick shaped repair. 

2. The database was sorted to retain only the repairs with masses greater than 0.004 lbm, the 
Orbiter deterministic foam debris limit. 

3. The maximum expected divot mass was then calculated using the maximum expected 
cylindrical defect 7 placed at either its maximum devoting depth or the depth of the repair, 
whichever was less. 

The result was a list of 86 divots from the 900+ repairs with masses ranging from 0.006 to 

0.028 lbm. 

The NESC was concerned that this calculation technique did not include the fact that repairs of 
less than 0.004 lbm can throw divots greater than 0.004 lbm owing to the angle of the divot cone. 
Also, it was not clear whether the possibility of slot defects had been included in the ET Project’s 
calculation. 

Therefore, the NESC repeated the ET Project analysis with two critical differences: 

1 . All PDL 1 034 repairs were evaluated for divoting. 

2. Both slot and cylinder voids were assessed. 

The resulting list of maximum expected slot and cylinder divots for each location was then 
sorted for the maximum expected (i. e., whether from a slot or cylinder). Of the 900+ PLD 1034 
repairs, 747 could generate divots larger than 0.004 lbm deterministic limit - a number 
substantially larger than the 86 generated by the ET Project. The NESC recommended to the 
Space Shuttle Systems Engineering and Integration (SE&I) Office that the NESC method be 
used to capture the maximum expected divot masses from the PDL 1034 repairs. The SE&I 
Office accepted the recommendation. 


7 The maximum expected defect is assessed from the available dissection data. The maximum void size observed in 
the dissections is increased by 40% to obtain the maximum expected defect. 
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Method 

The pressure difference between a sealed void and the external atmosphere cause a divot to occur 
during the Space Shuttle’s ascent. This void-delta p divoting mechanism was studied extensively 
during the post-Columbia return to flight effort. Divot/No-Divot curves were developed from 
test data from each type of foam on the External Tank that could be used to predict the maximum 
divoting depth for any sized void of the two characteristic void types, slots and cylinders. 

Once the 900+ PDL 1034 repairs were completed on ET 124, the length, width, and depth were 
known for each repair. This geometry information, the maximum expected size of cylinder and 
slot defects, plus the Divot/No-Divot curves could be used to calculate the maximum expected 
void size. 

The detailed methodology used for cylindrical defects was: 

1 . The cylindrical defect diameter taken as 1 .5 inches (the maximum expected diameter) or 
the largest repair dimension (length or width), whichever was smaller. 

2. The void depth was set at either the repair depth or its maximum di voting depth 
according to the cylinder Divot/No-Divot curve, whichever was smaller. 

3. The divot volume was calculated using a frustum with cone angle of 30° from the foam 
free surface as recommended by the ET Project. The minimum frustum diameter was the 
cylindrical defect diameter. 

4. The divot mass was calculated using the ET Project provided PDL 1034 density of 
3.44 lbm/ft 3 . 

The detailed methodology used for slot defects was: 

1 . The slot length was taken as largest repair dimension (length or width). 

2. The slot defect diameter was taken as 0.84 inches (the maximum expected diameter) or 

O 

the length/2.4, whichever was smaller . 

3. The void depth was set at either the repair depth or its maximum divoting depth 
according to the slot Divot/No-Divot curve, whichever was smaller. 

4. The divot volume was calculated using a translated frustum with cone angle of 45° from 
the foam free surface as recommended by the ET Project. The minimum frustum 
diameter was the slot defect diameter. The full length of the solid was the slot length. 

5. The divot mass was calculated using the ET Project provided PDL 1034 density of 
3.44 lbm/ft 3 . 

The cylinder and slot divot masses each PDL 1034 repair were then compared to determine 
whether the maximum divot mass was the result of a slot or cylinder defect. The largest mass 
was selected for reporting 8 9 . 


8 At length/diameter ratios less than 2.4, a slot is considered to be a cylinder. 
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Figure L-l plots the divot masses generated by the ET Project analysis and the NESC analysis. 
Even the figure shows that a very large number of possible divots greater than the deterministic 
limit of 0.004 lbm are not calculated are by the ET Project’s algorithm. 
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Figure 1-1 - Maximum Expected Divot Masses Using the Two Algorithms 

Figure L-2 plots sorted defect masses greater than 0.004 lbm obtained by the two algorithms. 
The figure shows the additional smaller masses between 0.004 and 0.010 lbm that are captured 
by the NESC method. 
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9 The slot void yielded the highest mass for only four of the repairs assessed, cylindrical defects yielded the largest 
masses for the remainder. 
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Figure L-2 — Sorted Divot Masses 

Using the NESC method allows all the maximum expected divots that exceed 0.004 lbm to be 
captured, generating the most accurate possible input to the PRA. The NESC recommended to 
the Space Shuttle Systems Engineering and Integration (SE&I) Office that the NESC method be 
used to capture the maximum expected divot masses from the PDL 1034 repairs. The SE&I 
Office accepted the recommendation. 
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Appendix J. Evaluation of the Inspection of Nose Cone, GH2 
Pressurization Line Fairing, and Intertank Access Door for ET 124 

Summary : 

Inspection of the ET 124 Composite Nose Cone at KSC using the MSFC Thermal Wave Imaging 
(TWI) thermography system was of better quality than the original inspection for manufacturing 
acceptance. This is because the original acceptance used a thermography system with a poorer 
resolution and sensivity. Inspectors were as well qualified and certified as those that performed 
the original acceptance inspection. Access to the Nose Cone was adequate to allow complete 
inspection. The critical initial flaw size (CIFS) was large in comparison to the damage in the 
NDE calibration impact damage coupons used to verify detection capability of the thermography 
system. During initial development, the capability of the Nose Cone to perform adequately with 
an included CIFS defect was verified by a damage tolerance test program prior to incorporating 
the Composite Nose Cone on the External Tank. Impact of a critical level would leave a visible 
dent on the surface of the Nose Cone. No visible indications were found upon visual 
examination of the Nose Cone. The thermography images from the inspection were reviewed an 
additional time by two more MSFC inspectors with no reportable indications found. 

Inspection of the GH2 Pressurization Line Fairing, and Intertank Access Door for ET 124 used 
hand held ultrasonic inspection. No review of data was possible as none was stored during the 
inspection. The initial acceptance inspection for the Pressurization Line Fairing was also hand 
held ultrasonic inspection. So the reinspection of this part was of similar quality to the initial 
acceptance inspection. However, the initial acceptance inspection of the Intertank Access Door 
uses a better quality inspection, C-scan ultrasound. This fact needs to be considered for 
relevance for the flight worthiness decision. 

Information on the inspection of the metal louver of the Nose Cone has been requested from 
Lockheed Martin MAF. An assessment of this inspection will be added at a later date. 

Supporting Material: 

The thermography images taken during the ET Nose Cone inspection were reviewed by Joseph 
Ragasa and Sam Russell on May 22 and 23, 2007. 

A Telecom with Carl Bouvier occurred on May 10, 2007 and James Walker joined me in my 
office. 


On the ET Nose Cone 

Mr. Bouvier and Mr. Walker conducted the thermography inspection on March 17 and 1 8, 2007. 
Mr. Ricky Clements represented SM&A and assisted in moving the inspection head and tripods. 
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Visual inspection of the three ET parts was performed by Kevin Vega (NASA), Ahmad Ekhlassi 
(USA), and Carl Bouvier (LM MAF) on March 6 2007. 

Mr. Bouvier has level III certification in ultrasonic and thermography inspection. He has passed 
a recent vision test using the Jaeger 2 at 12” or more method. 

The MSFC owned Thermal Wave Imaging (TWI) Flash thermography System was used. This 
system uses a FLIR Phoenix camera with 25 mm lens, TWI lamps, TWI computer, and TWI 
Echotherm 6.2 software used. Image subtraction of preflash was used. The Phoenix camera has 
thermal sensivity of 0.02 degrees K, and an Indium Antimonide 640 X 512 pixels array detector 
operating to detect 3-5 micro meter infrared light. Full sequence of tw2 images was stored at 
each inspection location. 

The Nose Cone had been dry dusted with a cloth prior to beginning inspection. 

No coating for emissivity control was used. The finish was the normal as processed finish. 

Metal tape indicators marked 7” x 7” grids. 

Four NDE coupons made from an early non-production Nose Cone were used to verify 
detectability of impact damage. These 3” x 3” coupons had been impacted by a cylindrical, 
round faced impacter at 3.3, 8.8, 5.2, and 6.6 ft lb of energy. Several other coupons were 
available that contained high energy impacted zones but were not used. The damage zones in the 
coupons were easily detected with the TWI thermography system. The impact at the 3.3 ft lb 
level specimen resulted in a barely visually detectable indention on the front surface. The 
damage zone on this specimen was measured by the Thermography system to be 0.49” on the 
front and 0.94” on the back in diameter. 

The coupons were inspected at different distances and up to a 60 degree angle to the normal and 
all damage zones were detectable. 

Initial acceptance of the Nose Cone was with thermography using an Inframetrics 760 camera. 

A set of heat lamps is positioned at 90 degrees to the Nose Cone with the Cone attached to a 
turntable. The Cone is rotated at about 1 rpm. This is somewhat equivalent to flash 
thermography. The Inframetrics 760 camera is a scanning system with a single Mercury 
Cadmium Telluride detector. The thermal sensivity is 0.1 degree K detecting 8-12 micro meter 
infrared light. 

Nose Cone is 1 8 plies thick at bottom, 24 plies thick at top, about % inches thick. 

On Pressurization Line Fairing 

Material is graphite epoxy. 6 to 10 plies thick. The part is approximately 12” long by 6” wide by 
6” high. 
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Inspected April 12, 2007 with hand scanned ultrasonics, with 5 MHz, 0.25” transducer by Carl 
Bouvier (LM MAF). Inspection was operated in pulse-echo mode off the back-surface using 
water squirter-bottle to control coupling. 

A defect standard of a fairing containing Teflon inserts was used to tune pulser/receiver. Set up 
with back surface Teflon inserts at 80% of full screen height (FCH). The base noise level is less 
than 10% FSH. 

No grid was used, just hand raster scan. The scan was done twice. The part was at head height 
when standing on small stool. Access to the part was unrestricted. The part flange could not be 
inspected because of interference of attachment bolts. 

Inspection was witnessed by Ricky Clements (MSFC) and James Walker (MSFC). 

Manufacturer used hand scan ultrasonic inspection for acceptance. Critical initial defect is %” x 
On Intertank Access Door 

Material is graphite epoxy, skin stiffened with bonded hat shaped vertical stringers. 

Inspected April 12 2007 with hand scanned ultrasonics, with 5 MHz, 0.25” transducer by Carl 
Bouvier (LM MAF). Inspection was operated in pulse-echo mode off back-surface using water 
squirter-bottle to control coupling. 

A defect standard of a section of door containing Teflon inserts was used to tune pulser/receiver. 
Set up with back surface Teflon inserts at 80% of full screen height (FCH). Noise level is 
unknown. 

No grid was used, just hand raster scan. The scan was done twice. Part was at knee to shoulder 
height. 

Inspection was witnessed by Ricky Clements (MSFC) and James Walker (MSFC). 

Manufacturer used C-scan ultrasonic inspection for acceptance. Critical initial defect is %” x 
Telecom with Bobby Biggs (LM) on May 14, 2007 

Discussion was on the development program for the Composite ET Nose Cone. 

A damage tolerance program was used to satisfy fracture control requirements. 

Flat coupon tests examined different types of damage. Impact was found to be the most 
significant damage in effecting compression strength. 

A 20 ft-lb impact with V” diameter cylindrical impacter was found to cause barely visible 
damage. 

Compression tests of coupons were conducted from -320 to 800 degrees F. The trend was for the 
impact to affect the compression property the same for all temperatures. 
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Full scale testing was conducted on cone with four 20 ft-lb impacts located in critical areas. This 
cone was tested to simulated loads for four flights lives and more than 3 times the maximum 
flight load once with no damage growth measured by ultrasonic inspection. 

A 20 ft-lb impact caused 2.5” diameter damage zone in coupons and 1” to 2” diameter damage 
zone in the full scale cone. 

1” by 1” critical defect size was developed from the damage tolerance program. 

May 17, 2007 


Sam Russell, Ph.D., P.E., ASNT LIN in UT, RT 
256 544 4411 

Nondestructive Evaluation Team, EM20 
NASA MSFC 
Huntsville, AL 35812 
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